Code Execution Agents

What code execution agents are

Most agents you have already met take action through function calls: the model writes a tidy JSON blob saying “call search_database with these arguments,” and your application is responsible for dispatching that call, formatting the result, and feeding it back. A code execution agent flips this. The model writes a short program — usually Python — and a sandbox actually runs it. The return value becomes the next observation in the reasoning loop. The action space is the language itself.

JSON tool calls compose badly. If you need to fetch three rows, join them on a key, filter by date, and average a column, the agent has to chain four or five separate tool calls, with the model re-reading intermediate JSON each time. A few lines of Python does the same work in one step and one round trip. According to the CodeAct paper (Wang et al., ICML 2024), this pattern produced up to 20% higher task success and roughly 30% fewer steps versus JSON-style actions across 17 evaluated LLMs.

The runtime side matters as much as the model side. Because the agent is generating real code, that code has to execute somewhere that cannot harm the host machine. Modern implementations rely on isolated sandboxes — Firecracker microVMs, container-based environments, or vendor-managed interpreters — that boot fast, expose a controlled filesystem, restrict network access, and tear down cleanly between sessions. The pairing of “code as the action format” plus “interpreter that is safe to run untrusted code in” is what makes the whole pattern viable in production.

How code execution agents are used in practice

The most common entry point for a non-engineer is a chat interface that quietly runs Python for you. When you upload a CSV to ChatGPT and ask “which region had the largest drop in conversions last quarter?”, a code execution agent is what answers. The model writes a pandas snippet, the sandbox runs it on your file, and you get back a chart and a sentence. According to the OpenAI API Docs, this lives under the Code Interpreter tool on the Responses API. According to the Claude API Docs, Anthropic exposes an equivalent code execution tool that runs Python and Bash in a sandboxed container.

Developers building products on top of these APIs use the same pattern for data analysis features, document parsing, on-the-fly calculations, generating downloadable files, and any task where the cleanest expression of the work is a few lines of code rather than a forced sequence of JSON calls.

Treat the sandbox as a hostile environment that happens to live inside your product. Set per-call timeouts, cap memory, restrict network egress to an explicit allowlist, and never share a sandbox session across two end users. Most production incidents with code execution agents come from skipping one of these four steps.

The code execution agent reading path: mechanism, limits, then the build

Start with how sandboxed interpreters let LLMs run their own code — it explains why letting the model write and execute Python inside a container often beats structured tool calling, and what a minimal stack looks like. Then read the prerequisites, from ReAct loops to microVM isolation: it lays out the three layers — reasoning loop, sandbox runtime, hardware isolation — that have to exist before any of this is safe, which is what separates a toy demo from a defensible security posture.

Before you build anything, the honest limits read is worth the ten minutes: cold-start latency, flaky benchmark tests, and shrinking effective context are structural, not vendor-specific, so budget for them rather than debug around them later. When you’re ready to wire one up, the E2B, Daytona, and Claude Agent SDK guide specifies the sandbox contract — lifecycle, network rules, timeouts — before a single line of tool code. For the market context behind that choice, the 2026 SWE-bench race coverage shows why the harness wrapped around a model now moves benchmark scores more than swapping the model itself. Close with the accountability question — if your agent will ever touch production credentials, read it before you ship, not after.

How code execution agents differ from tool calling, frameworks, and retrieval

Three neighbours get folded into this topic, and each mix-up sends the design in the wrong direction.

• Code execution is not JSON tool calling. Classic tool use has the model emit a structured call, wait for a round trip, then decide again — one LLM turn per step. A code execution agent writes a script that chains several operations itself, calling the model only when it needs a new decision. The gain is fewer round trips; the cost is a much bigger blast radius per turn, which is exactly what the sandbox layer exists to contain.
• The sandbox is not the agent framework. Agent frameworks like LangGraph or CrewAI decide how steps are sequenced and which tool gets called next; the sandbox is what actually executes the code once the framework hands it off. Swapping frameworks doesn’t change your isolation posture, and hardening the sandbox doesn’t fix a broken control flow — they’re separate contracts.
• Running code is not retrieving knowledge. Retrieval-augmented agents decide when and what to search for; code execution agents decide when and what to run. Some production systems need both loops side by side — a retrieval decision to find the right data, then a code decision to transform it — and conflating the two is a common source of unpredictable latency.

When code execution agents fit — and when they don't

Common questions about code execution agents

Q: How is a code execution agent different from a tool-calling agent that returns JSON? A: A tool-calling agent emits one structured call per LLM turn and waits for the result before deciding again. A code execution agent writes a script that chains several operations in one pass, calling the model only when a new decision is needed — fewer round trips, but more damage per turn if the sandbox isn’t isolated. See how sandboxed interpreters let LLMs run their own code.

Q: Is the Claude Agent SDK’s built-in Bash tool safe to use in production? A: No — it runs inside your own process, not a sandbox, so model-written code executes with your process’s own permissions. Use it for local development only; in production, replace it with a custom tool that routes execution through E2B or Daytona, as the build guide specifies.

Q: Do I need microVM isolation, or is a container sandbox enough? A: It depends on what the model is allowed to do: a standard container shares the host kernel, fine for trusted, narrow tasks, but model-written code is untrusted by definition. The prerequisites read covers why production stacks add a hardware-level boundary like Firecracker or gVisor underneath.

Q: Should I evaluate a code execution agent by its base model or by its scaffold? A: By the scaffold first. The same underlying model can post very different SWE-bench results depending on the harness wrapped around it, so a leaderboard comparison across vendors is really a comparison of engineering, not just model quality. The 2026 SWE-bench race breaks down what shifted.

Q: Who is accountable when a code execution agent breaks something in production? A: Legally and organizationally, nobody has fully answered that yet — the credentials were authorized, but not for the specific action the agent took. The accountability piece traces a real incident where an agent deleted a production database during a declared code freeze.

Part of the AI agent architecture theme · closest neighbour: agent planning and reasoning. Coming to sandboxed execution from a software background? Start with the story: Debugging Agents: Reconstruct the Decision Path, Not a Stack Trace.

Sources

• the CodeAct paper: Executable Code Actions Elicit Better LLM Agents - Foundational ICML 2024 paper that formalized code-as-action and reported the headline gains over JSON tool calls.
• Claude API Docs: Code execution tool - Reference for Anthropic’s sandboxed Python + Bash execution capability for Claude models.