Opro
Also known as: Optimization by PROmpting, LLM-as-optimizer, automatic prompt optimization
- Opro
- OPRO (Optimization by PROmpting) treats an LLM as an optimizer: it receives a history of candidate prompts with performance scores and generates new, improved prompts — automating iterative refinement without requiring gradient access or model fine-tuning.
OPRO is a technique where a large language model acts as its own prompt optimizer, reading a history of candidate prompts with performance scores and generating improved versions until performance stabilizes.
What It Is
Manual prompt refinement is slow. Engineers write a prompt, run it on test cases, note where it fails, revise the wording, and repeat. For tasks with measurable outputs, this cycle is exactly the kind of iterative search that should be automated. OPRO (Optimization by PROmpting) does that: it turns prompt refinement into a language task that an LLM runs on your behalf.
Think of it like showing a copywriter a log of past ad headlines along with their click-through rates and asking them to write the next version. They don’t need the ad algorithm — they read the evidence and adapt. OPRO works the same way: the optimizer LLM reads a context window filled with (prompt, score) pairs and generates a new prompt it predicts will score higher.
The loop runs in cycles:
- Start with one or more seed prompts
- Evaluate each on a validation set, producing a numeric score
- Add the (prompt, score) pair to a growing history
- Pass the history to the optimizer LLM with an instruction to write a better prompt
- Evaluate the new candidate, add to history, repeat
- After a fixed number of iterations, return the highest-scoring prompt found
In the prompt optimization context this entry supports, OPRO sits between manual refinement and systems like DSPy. Manual refinement relies entirely on human judgment. DSPy compiles a full pipeline — selecting examples, generating chain-of-thought traces, optimizing module by module. OPRO covers a narrower scope: the wording of a single instruction prompt, optimized by the LLM’s language understanding rather than by gradient descent or human trial-and-error.
Because OPRO doesn’t need model internals — no gradients, no logit outputs, no weight access — it works with any model accessible through an API. You can use one LLM as the optimizer to improve prompts for a different LLM running the actual task, as long as you can measure the output quality.
How It’s Used in Practice
The most common scenario: a team has a task-specific LLM application — a classifier, a data extractor, a summarizer — that produces inconsistent results. They have labeled examples but no budget or access for fine-tuning. OPRO automates the search for better instructions by treating the optimization as a language task.
In practice, you define a scoring function (such as exact-match accuracy on a validation set), provide the optimizer LLM with the history of scored attempts, and let it run for several iterations. The output is a candidate prompt that performed well on your examples — not guaranteed optimal, but consistently better than the starting point.
A less obvious use: diagnosing whether the current prompt is already near its ceiling. Running OPRO for a fixed number of iterations with no meaningful score improvement signals that the prompt wording itself isn’t the bottleneck — the task definition, the model, or the data quality may be the limiting factor.
Pro Tip: OPRO works best when you have a clear, measurable scoring function — even a simple one like exact-match accuracy on 20–50 examples. Fuzzy goals like “sounds more professional” are hard to score reliably, which makes the optimization loop converge slowly or produce misleading results.
When to Use / When Not
| Scenario | Use | Avoid |
|---|---|---|
| You have labeled examples and a clear accuracy metric | ✅ | |
| Prompt quality is subjective and hard to score numerically | ❌ | |
| You want automated improvement without changing model weights | ✅ | |
| You need real-time optimization during production inference | ❌ | |
| Your validation set is fewer than 20 examples | ❌ | |
| You’re refining instructions for a specific recurring task | ✅ |
Common Misconception
Myth: OPRO replaces the need for any human judgment in prompt design.
Reality: OPRO automates the search within a space of prompt wordings, but humans still define the task, build the scoring function, and validate whether the winning prompt generalizes to real production traffic. A poorly constructed scoring function produces a prompt that looks optimized on paper and fails in use.
One Sentence to Remember
OPRO doesn’t replace prompt engineering — it automates the iterative trial-and-error part, so engineers can focus on defining what good performance actually means rather than cycling through wordings by hand.
FAQ
Q: How is OPRO different from DSPy? A: Both automate prompt optimization, but DSPy compiles full pipelines including few-shot examples and chain-of-thought traces. OPRO focuses specifically on instruction text, using the LLM’s own language understanding to propose improved wordings based on past scores.
Q: Does OPRO work with any LLM, including API-only models? A: Yes. OPRO needs only a way to score prompt outputs — no model gradients or internal access required. You can use one LLM as the optimizer and a completely different one as the task model.
Q: How many iterations does OPRO typically need? A: Performance usually stabilizes within a few dozen iterations. The main limiting factor is API cost and dataset size, not the algorithm — start small and increase the iteration budget only if the score keeps improving.
Expert Takes
OPRO reframes prompt optimization as a pattern-completion problem. The optimizer LLM reads a context of (instruction, score) pairs — analogous to a few-shot classification task — and generates a next instruction token-sequence predicted to yield a higher score. What appears to be adaptive reasoning is the same in-context learning mechanism that makes LLMs useful elsewhere, applied recursively. The technique does not require the model to understand the task — only to recognize patterns in performance data.
The practical case for OPRO is that it fits inside an existing prompt workflow without new infrastructure. You need a validation set and a scoring function — both things a production team should have regardless. OPRO then runs as an offline job: feed in the examples, let it iterate, deploy the winning prompt as a system prompt update. No fine-tuning pipeline, no gradient access, no model contracts to renegotiate. The output is a plain text string.
Manual prompt tweaking has been the default because it looked cheaper than building an optimization pipeline. OPRO closes that gap. If your task has measurable outputs and a labeled test set, running an optimization loop costs a fraction of the engineering time that manual iteration demands — and it doesn’t depend on one engineer’s intuition about what sounds right. Teams still treating prompt search as a craft exercise are leaving measurable accuracy gains on the table.
There is a question OPRO’s framing sidesteps: who is actually optimizing? The LLM reads scored prompts and generates the next candidate — but it is completing a pattern in context, not reasoning about the task. When that loop runs on systems affecting real decisions — hiring, lending, triage — the line between “the model optimized this” and “a developer chose this” blurs in ways that matter for accountability.