What Is Prompt Engineering and How Zero-Shot, Few-Shot, and Chain-of-Thought Techniques Control LLM Output

Adding the phrase “Let’s think step by step” to an InstructGPT prompt shifted MultiArith accuracy from 17.7% to 78.7% — no retraining, no additional data, just seven words (Kojima et al. 2022). That result demands an explanation. If a purely verbal intervention can shift model accuracy by that magnitude, something structural must be happening inside the inference process. The surface explanation — “better prompts produce better outputs” — describes the observation without touching the mechanism.

The Input That Changes the Distribution

A language model doesn’t retrieve answers from a lookup table. It generates each output token by computing a probability distribution over its entire vocabulary, conditioned on every token that preceded it. Everything inside the Context Window participates in that conditioning — not as passive data, but as an active constraint on what tokens the model assigns high probability to next. Change the constraints, and you change the distribution; change the distribution, and you change what gets sampled. The prompt is not a suggestion — it’s a boundary condition.

What is prompt engineering?

Prompt engineering is the practice of designing text inputs — including context, instructions, examples, structural cues, or any combination — that shift a language model’s output distribution toward a specified target. No parameter updates occur. The model’s weights remain fixed; what changes is the conditional distribution P(output | input), shaped entirely by the structure and content of the input provided at inference time.

A systematic survey catalogued 58 distinct LLM prompting techniques organized across six categories (Schulhoff et al. 2024). That breadth is a signal: there is a structured relationship between input form and output behavior that admits systematic exploitation, not just intuition-driven trial and error.

The leverage point is Instruction Following capacity. Models trained with reinforcement learning from human feedback are specifically optimized to respond to task framing, making zero-shot performance substantially better than earlier pre-RLHF models. But instruction-following still means sampling from a probability distribution shaped by those instructions — the probabilistic substrate never disappears.

Separating behavioral constraints from user-facing inputs via a System Prompts structure stabilizes the distribution across interactions. The system prompt anchors output style and task framing even as user messages change, functioning as a persistent prior that conditions every subsequent generation.

How does prompt engineering change what a large language model outputs?

The mechanism operates through the attention architecture’s ability to weight relationships between all tokens in context simultaneously. When you structure an input a particular way, you activate different attention patterns across the network — not by rewriting the weights, but by determining which patterns the fixed weights make relevant for this particular conditioning.

Not magic. Conditional probability.

Three techniques illustrate how different constraints produce different distributional effects:

Zero Shot Prompting provides no examples; the model relies on patterns absorbed during pretraining to infer what the task requires. The full prior distribution is in play. For tasks with broad training coverage, this is sufficient. For tasks requiring a specific output format, a particular reasoning style, or domain-specific framing, the prior contains too much entropy — too many equally probable continuations exist and the model selects among them with no additional constraint.

Few-Shot Learning embeds labeled examples directly in the input. When Brown et al. demonstrated in-context learning in 2020, the key result was precise: a 175-billion parameter model could perform tasks by receiving labeled demonstrations embedded in the prompt, with no gradient updates or parameter changes required (Brown et al. 2020). The examples don’t teach the model anything in the traditional sense. They shift the conditional distribution by providing observed cases that concentrate probability mass around outputs matching the demonstrated pattern — Bayesian posterior conditioning over sequences, without weight updates.

Chain-of-Thought prompting embeds intermediate reasoning steps alongside examples. Wei et al. found in 2022 that this substantially improves performance on complex reasoning tasks, particularly multi-step arithmetic and symbolic problems (Wei et al. 2022). The plausible mechanisms are several: intermediate tokens may keep relevant information inside the attention window; the chain structure may activate reasoning-adjacent patterns from pretraining; or the format may constrain the sampling path such that intermediate tokens must be internally coherent before the final answer is sampled. The effect is reproducible. Which explanation is correct — or whether all three contribute — remains an active research question.

Conditioning at Scale: Technique, Threshold, and Interaction

The three techniques are not interchangeable; they interact with model scale and task structure in ways that determine when each is appropriate.

Zero-shot prompting via Role Prompting — assigning a persona or epistemic stance at the start of a prompt — can shift vocabulary, register, and apparent framing without any examples. This works because role assignment functions as a distributional prior: “You are an expert in X” changes which tokens are probable next by activating patterns from the pretraining distribution associated with expert-register discourse in that domain. Whether the model actually reasons differently in such a role is a separate question; it samples differently, which changes what it produces.

Few-shot prompting works best with three to five examples wrapped in structured markup; examples should be relevant to the target task, diverse enough to demonstrate the range of expected inputs, and formatted consistently (Anthropic Docs). The structure matters because inconsistent formatting across examples introduces ambiguity into the conditioning signal — spreading probability mass across multiple output patterns rather than concentrating it.

Format signals matter as much as content. Distributional noise introduced by inconsistent markup in the few-shot demonstrations degrades the conditioning signal in proportion to the inconsistency.

Meta Prompting — instructing the model to generate, evaluate, or refine its own prompts — introduces a recursive conditioning layer. The model’s output in one pass becomes the input constraint for the next. This amplifies the sensitivity of the approach: a well-constructed meta-prompt compounds across iterations, and a poorly constructed one introduces distributional drift that accumulates with each cycle.

What the Mechanism Predicts — and Where It Fails

Understanding that prompts function as inference-time distributional constraints makes several predictions explicit.

If you change the few-shot examples while holding the instruction constant, you should observe output format shifts — because the examples are the primary distributional signal for format, not the instruction text. This means example selection carries as much weight as instruction clarity.

If the task involves many intermediate reasoning steps, chain-of-thought conditioning should help — unless the model has built-in reasoning that already handles intermediate steps internally. As of mid-2026, extended thinking modes in models like Claude Fable 5 and the o-series from OpenAI handle intermediate reasoning natively. A recent analysis found that chain-of-thought prompting yields marginal or no accuracy improvements on models with built-in reasoning, while significantly increasing token consumption (Meincke et al. 2025). The technique that outperformed fine-tuning in 2022 now actively wastes tokens on reasoning-native models. Whether to include explicit chain-of-thought is now a question about the model’s architecture, not just the task’s complexity.

If the system prompt contains sensitive instructions, those instructions exist inside the same conditioning context as everything else the model processes. The model has no native mechanism to mark some tokens as private and others as public. Prompt Leakage — the extraction of system prompt content through targeted user inputs — is a direct consequence of this architectural flatness.

The same property that makes conditioning powerful makes it exploitable. Prompt Injection attacks introduce malicious instructions into the model’s context through untrusted inputs — a retrieved document, a user message, a tool response — exploiting the model’s inability to distinguish trusted instructions from untrusted content when both arrive as the same token stream. In agent frameworks where models read external data sources and execute downstream actions, this attack surface is substantive and has been demonstrated with real consequences.

Rule of thumb: If you cannot predict how your system’s output distribution would shift if an adversarial string were inserted into the context, your system has undefined prompt injection exposure.

When it breaks: Prompt engineering has no reliable mechanism for preventing a sufficiently capable adversarial input from overriding conditional constraints set by the original prompt. The distributional shift produced by a hostile injection can exceed the distributional shift produced by the legitimate system prompt — there is no architectural guarantee of precedence.

Security & compatibility notes:

• Prompt Injection → RCE in Agent Frameworks (CVE-2026-25592, CVE-2026-26030): Prompt injection vulnerabilities in AI agent frameworks (Microsoft, May 2026) enable remote code execution in systems where model outputs trigger external actions. Apply vendor patches; treat all external data entering the context as untrusted; architectural separation of instruction and data streams is required.
• OpenAI Prompt Objects API (Deprecated): The v1/prompts endpoint was deprecated June 3, 2026; full shutdown scheduled November 30, 2026. Developers using managed prompt objects must migrate prompts into application code.

The Signal That Context Engineering Names

The move from prompt engineering to Context Engineering is not a rebranding exercise. It reflects a structural shift in what the problem actually is.

Andrej Karpathy’s June 2025 framing — “the delicate art and science of filling the context window with just the right information” — names something that zero-shot, few-shot, and chain-of-thought prompting all assume: that the right information is determined at prompt-design time (X, Karpathy). In agentic systems, it isn’t. The context is populated dynamically by tool outputs, retrieved documents, conversation history, intermediate reasoning, and external data. The question shifts from “what should I write in the prompt?” to “what architecture determines what enters the context, in what form, and in what order?”

Prompt engineering remains the foundation. The techniques work because the mechanism works — conditional probability conditioning via input structure is real, reproducible, and measurable. What has changed is the scale of what lives inside the context window, and the degree to which what enters it is under deliberate architectural control rather than direct human authorship.

The Data Says

Prompt engineering works because prompts are not instructions received by a thinking entity — they are constraints imposed on a sampling process. Zero-shot, few-shot, and chain-of-thought prompting each alter the conditional distribution P(output | input) in structurally distinct ways, without modifying model weights. The calculus has shifted: for models with built-in reasoning as of mid-2026, chain-of-thought prompting adds token cost without proportional accuracy gains (Meincke et al. 2025), and the real engineering question is now which information enters the context window, not just how the instruction is phrased.