Agent Evaluation and Testing

What agent evals are

According to Google Cloud Docs, a trajectory is the sequence of tool calls an agent took to reach the final response. That single definition explains why agent evaluation differs from evaluating a chatbot: there is now a path to grade, not just an answer. An agent that books your flight by calling the wrong API in the wrong order — but happens to land on the right itinerary — has produced a correct outcome through a broken trajectory. Evaluation has to catch both shapes of failure.

The discipline emerged because production teams kept hitting the same wall. A model that answers questions accurately in isolation behaves unpredictably once it can plan, call tools, and chain steps. According to the LangChain State of Agent Engineering, quality has become the leading deployment barrier for organizations running agents in production. Teams needed a measurement layer deeper than “did the response look right.”

Modern agent evaluation works on two complementary layers. The outcome layer scores final state: did the database get updated, did the email send, did the answer match the gold reference? The trajectory layer scores the path: which tools were called, in what order, with what arguments, and was that the right way to get there? According to Anthropic Engineering, an evaluation harness orbits around a small set of primitives — Task, Trial, Grader, Transcript, Outcome — and you grade trajectories by feeding the trace through automated checks, LLM-as-judge graders, or both.

Trajectory grading is rarely binary. According to LangChain’s GitHub repository, the AgentEvals library exposes four match modes: strict (exact ordered sequence), unordered (same tools, any order), subset (expected calls are contained in the actual run), and superset (actual calls are contained in the expected set). Different agents need different strictness — a research agent benefits from unordered matching, a payment agent needs strict ordering. Picking the right match mode is half the design work.

How agent testing is used in practice

The most common entry point is connecting an agent framework to an evaluation platform during development. A product team building a customer-support agent on top of Claude or GPT will instrument runs through LangSmith, Vertex AI’s evaluation service, Arize Phoenix, or Braintrust. The platform captures every tool call, system prompt, and intermediate response as a trace. From there, the team writes a fixed dataset of test conversations, runs the agent over them in CI, and inspects which outcomes regressed and which trajectories drifted between releases.

The second common scenario is production monitoring. Once the agent ships, traces from real user sessions stream into the same evaluator, where trajectory rules and LLM-as-judge graders flag suspicious patterns — repeated tool retries, hallucinated function arguments, broken plan-and-execute loops. Evaluation becomes a continuous process rather than a one-time gate.

Build your trajectory dataset from real production traces before writing synthetic ones. Synthetic test cases capture what you imagine the agent will face; production traces capture what it actually faces, including the edge cases you never thought to write.

How to read agent evaluation and testing

Start with what trajectory analysis actually scores — it explains why grading the sequence of tool calls catches failures a single accuracy number hides. Read the prerequisites — LLM-as-judge, trajectory scoring, and cost-per-task limits next: it names what each of those three measurement layers can and cannot tell you before you commit to a metric.

When you’re ready to wire it up, the pipeline guide for LangSmith, Braintrust, and DeepEval turns the three prerequisites into a CI layer, an experiments layer, and a production trace layer that run together. For where the vendor landscape is moving, the read on Maxim, Galileo, and Laminar explains why agent-first eval platforms are pulling share from generic observability tools. Close with when agent evals lie — if LLM-as-judge scoring sits at the center of your pipeline, read this before you present its numbers as ground truth.

How agent evaluation differs from guardrails, cost limits, and recovery

• Evaluation measures, guardrails enforce. Agent guardrails block actions at runtime before they execute; evaluation runs offline or in staged experiments to score whether the agent that already ran did the right thing. Airtight guardrails and a broken eval pipeline can coexist for months before anyone notices.
• A cost-per-task ceiling is a number evaluation reports, not a lever it pulls. Agent cost optimization is the discipline of actually lowering that number through routing, caching, and token budgets; evaluation’s job stops at telling you the number crossed the line.
• Evaluation flags the regression; it doesn’t fix the run. Agent error handling and recovery is what the agent does in the next step when a trajectory eval keeps catching the same tool-call failure pattern — the eval is the signal, not the retry.

When AI agent evaluation fits — and when it doesn't

Common questions about agent evaluation and testing

Q: Can I evaluate an agent with the same test suite I use for regular software? A: Not as-is. Unit tests check a fixed input against a fixed output; agents take a different path each run, so trajectory analysis grades the sequence of decisions and tool calls instead of a single expected string.

Q: Is a high task-completion rate enough to trust an agent in production? A: No — completion says the agent finished, not that it finished correctly or affordably. The evaluation prerequisites explainer names trajectory correctness and cost-per-task as the two signals completion rate leaves out.

Q: Do I need a dedicated evaluation platform, or can spreadsheets and manual review get me started? A: Manual review works for a handful of test cases, but it doesn’t scale past a demo. The pipeline guide shows why teams end up running a CI layer, an experiments layer, and a production trace layer together instead.

Q: Can I rely on LLM-as-judge scoring as my primary evaluation signal? A: Only with caution — a judge model can carry the same biases and blind spots as the system it grades, and a wrong verdict looks exactly as confident as a correct one. The accountability risks of LLM-as-judge scoring covers what that means for numbers you report upward.

Q: Should I trust vendor benchmark rankings when choosing between agent evaluation platforms? A: Treat them as a shortlist, not a verdict — the market is moving fast enough that agent-first platforms like Maxim, Galileo, and Laminar are actively displacing incumbents your benchmark comparison may not reflect yet.

Part of the agent reliability and operations theme · closest neighbour: agent guardrails. Coming to this from an SRE background? Start with the story: Agent Reliability for Engineers: What SRE Habits Map and Break.

Sources

• Google Cloud Docs: Evaluate Gen AI agents — Vertex AI documentation - Reference for trajectory definition and match metrics.
• Anthropic Engineering: Demystifying evals for AI agents - Eval harness primitives and grading patterns.