Debugging Agents: Reconstruct the Decision Path, Not a Stack Trace

The feature worked in the demo. In production, an agent-backed workflow started returning the wrong answer for about one request in fifty — a refund quoted against the wrong policy, a summary citing a document nobody uploaded. You pull the run. Forty steps, every one green. Every tool call logged a 200. The planner emitted valid actions. The final model call returned clean, well-formed JSON. Nothing threw. There is no red line to scroll to, because nothing failed the way failure is supposed to look. The bug is real and repeatable enough to matter. It just isn’t located anywhere your debugger can point.

When you call a model in a loop, you don’t inherit a program you can step through — you inherit a decision path, and debugging it means reconstructing why locally-correct steps reached a globally-wrong answer. The reflex that betrays you first is the most trusted one: that the trace is a stack trace and the fault sits at the last call that threw. Nothing threw. This is orientation for engineers who wire up agents, not the ones who train them.

An Agent Run Is a Distributed Trace

Start with what still transfers, because a lot of it does. You have debugged systems where the failure wasn’t in any single service — a request fanned out across six of them, each returned 200, and the user still got the wrong total. You already know that move: read the run as a trace, follow the correlation ID from hop to hop, and find the point where a locally-valid response was the wrong response to hand downstream. An agent run is that kind of trace. The planner, the tool calls, the state reads, the sub-agent hand-offs — they are hops, and treating the whole run as one distributed transaction is the correct instinct.

Here is where the map needs an extension, because this is exactly where the instinct breaks down. A stack trace points at the frame that threw. A distributed trace at least points at the service that returned bad data. An agent run points at nothing, because the fault is a decision, and decisions don’t raise exceptions. The model chose to call the refund tool on a request it should have skipped. It chose to trust a retrieved document that didn’t answer the question. Each choice was well-formed. None of them is an error your instrumentation was watching for. The real fault usually sits several hops upstream of the symptom — in the reasoning step that picked the wrong branch, not the tool call that faithfully executed it.

That changes what you log. HTTP status and schema validity tell you the plumbing held. To debug the decision path you have to capture the inputs and outputs of each reasoning step — what the model saw, what it decided, and what the next step inherited as ground truth. The pattern you picked for coordination decides how legible any of this is; Mona’s breakdown of graph, conversation, and crew architectures makes the case that your framework choice is really a choice about which class of failure you will spend next quarter debugging.

Instrument the decisions, not just the calls. A run where every span is green and the outcome is wrong is not a passing run — it’s an under-instrumented one.

Mental Model Map: Debugging an agent run From: A run is a stack trace; the fault is the last call that failed. Shift: No call failed — the fault is a decision several hops upstream that looked correct when it ran. To: A run is a decision path; you debug it by reconstructing why locally-right steps summed to a wrong result. Key insight: The step that broke the run raised no error, so the trace never points at it.

Most of your on-call reflexes survive the move — they just stop predicting at a different point than they used to. Here is where each one holds and where it quits:

Every Step Passed. The Run Failed.

The reason there’s no failing assertion to anchor on is arithmetic. Chain enough steps together and reliability stops being additive. An Agent Planning And Reasoning loop that is right ninety-five percent of the time at each step is not ninety-five percent reliable end to end — string twenty of those steps together and the whole chain lands around a third, because the errors multiply. Mona’s analysis of where each planning pattern’s ceiling lives puts the harder number on it: past roughly a hundred and twenty sequential steps, even the strongest models’ accuracy approaches zero. No single step has to be broken for the run to be. The failure is spread across the path, which is exactly why no per-step check catches it.

A second trap waits when you try to reconstruct the path from the model’s own words. Modern agents narrate their reasoning — “I chose this tool because the policy was ambiguous” — and it is tempting to read that narration as a log. It isn’t one. Studies now show the printed chain of thought is often not a faithful account of what the model actually used: in one study of a reasoning model, it acknowledged a planted hint in almost every case yet admitted to using it in under two percent of them. So the reasoning text is a plausible story assembled after the fact, not an execution record. Read it as a lead, then confirm the decision against the things that don’t editorialize — the exact inputs the step received, the state it read, and the output it produced.

Your evidence is the state and the I/O, not the model’s self-report. When those two disagree, believe the state.

Retrying Rolls the Dice Again

The oldest reflex on the list is the most dangerous one here: see a failure, retry it. In a deterministic system a retry either reproduces the bug or clears a transient blip, and both outcomes teach you something. Retry a non-deterministic agent and you learn almost nothing. The reproduce-first assumption breaks the moment the path turns non-deterministic. The same input can send it down a different decision path — one benchmark that asked agents to solve the same task eight times in a row saw success fall under a quarter, and that gap is the whole problem. A green re-run doesn’t mean you fixed anything. It means the dice came up different.

Worse, the re-run isn’t free. The first attempt already sent the email, wrote the row, charged the card. Retry without guarding those actions and you don’t debug the incident — you cause a second one.

The instinct that does partly transfer is checkpointing. A stateful agent is a stateless model wrapped in a serialization loop: Agent State Management tooling like LangGraph writes the run’s state after each node, so you can resume from a saved point instead of restarting. Mona’s walkthrough of threads and checkpointers shows how far the bisect-from-a-checkpoint habit carries — and where it stops.

Replaying a checkpoint restores the state, not the model’s sampling, so the resumed run can still fork. And durability stops at the node boundary; the retry logic and recovery scope you actually get depend on whether you built on a Workflow Orchestration For AI engine that treats the whole run as durable, which Mona’s comparison of DAGs and state machines lays out step by step.

A retry that re-fires side effects and takes a fresh path isn’t a retry — it’s a rerun with consequences. Make every write idempotent, gate irreversible actions behind an explicit check, and log the path each attempt took, so you can tell a real fix from a lucky draw.

The Path Forks Where No Trace Follows

Reach for throughput and the debugging problem grows a dimension. Your distributed-systems instinct says split the work: more workers, more parallelism, faster runs. With Multi Agent Systems that instinct is half-right and half a trap. When Cognition’s team split a small build across two parallel sub-agents, the pieces didn’t fit — one wrote a Mario-style background, the other a bird that didn’t match — because the sub-agents shared filtered messages instead of full traces. Each made implicit decisions the other never saw. The decision path forked, and no single trace spanned the fork, so there was nothing to reconstruct afterward.

The counter-example proves the same point from the other side. Anthropic’s orchestrator-worker setup beat a single agent by about ninety percent on its internal research eval — but only by sharing context aggressively and paying for it in tokens, and only when the task actually decomposed. The prerequisites-and-limits breakdown for multi-agent orchestration and the supervisor, debate, and swarm architecture guide both land on the same rule: coordination is a control-graph decision, and the graph decides whether a failure is traceable at all.

This is also where a wrong run hides in plain sight. An orchestrated workflow can stay green end to end and still hand a customer the wrong outcome, and Alan’s piece on how orchestration hides the failure is worth reading before you decide a passing pipeline means a correct one. Decide your trace-sharing contract up front: a hand-off without a shared trace is a blind spot you are choosing to build.

Shift Diagram: How the investigation runs Classic: crash → open stack trace → failing frame → fix the line AI: wrong outcome → reconstruct decision path → upstream reasoning step → fix the spec or state

Before You Put an Agent Run On Call

Before an agent-backed workflow carries real traffic, these are the questions to ask about your own stack — not googleable trivia, but the ones that decide whether you can debug the next incident at all.

For the full prerequisite map and the questions that sit behind these, see the topic hub.

An agent run is not a program that throws — it’s a decision path that drifts, and the fault is almost never the last step you can see. Stop debugging from the bottom of the trace. Instrument every decision, pin what you can, gate every irreversible write, then reconstruct the path from the top. Start Monday by adding input-and-output logging to each reasoning step in one workflow you already run.