Prompt engineering that actually works

By Ryze AI Team

Most prompt advice on the internet is folklore passed around until it sounds like fact. The techniques that actually hold up under measurement are unglamorous: be specific, show examples, constrain the output, and iterate with a fixed evaluation set. Everything else is decoration.

Be specific about the task

Vague prompts produce vague results. State the role the model should take, the input it will receive, the exact output format you want, and the constraints. Compare "summarize this" with "summarize this article in three bullet points, each under fifteen words, focusing on actionable takeaways for a software engineer." The second gives the model a measurable target. Include what not to do as well: "do not include background information" or "do not speculate beyond the text."

Few-shot examples

Two to five input-output pairs anchor the model's behavior far better than instructions alone. The model infers the pattern from the examples, including tone, length, and formatting, and applies it to the new input. Choose examples that cover the normal case and the edge cases you care about. Keep them consistent in format; an inconsistent example teaches the model that format does not matter. More examples help up to a point, after which you are spending tokens on diminishing returns and crowding out the actual input.

Constrain the output

Ask for JSON with a schema, use delimiters between sections, and specify field names. Constrained decoding, available in many runtimes, guarantees valid structure by masking invalid tokens during generation. For models that do not support it, ask the model to output only the JSON and parse defensively. Constrained output is the difference between a demo and a production system, because downstream code can rely on the shape.

Chain of thought, used carefully

Asking the model to reason step by step improves accuracy on math, logic, and multi-step problems. It also adds tokens, latency, and cost, and can make the model over-explain simple tasks. For small models, a shorter prompt with good examples often beats a long chain-of-thought prompt. Use chain of thought when the task genuinely requires intermediate reasoning, and ask the model to put the reasoning in a separate field you can discard, keeping the final answer clean.

Iterate with evals

Change one thing at a time and measure on a fixed set of inputs with known good outputs. Keep a prompt log with scores so you can see what worked and what did not. What worked last week may stop working after a model update, so re-run your eval set whenever you change the model or the prompt. Treat prompt engineering as an experiment, not a craft; the numbers decide, not your intuition.

Common mistakes

Over-prompting: giant system prompts that contradict themselves and dilute attention. Under-specifying the output format, then parsing with fragile regex. Tuning on a single example and calling it done. And copying a prompt from a blog post without adapting it to your model and task. Prompts are not portable across models; a prompt that shines on GPT-4 may be mediocre on a 7B local model.

Resources: Prompting Guide and the model provider's own prompt documentation.