Unverifiable Actions: Accountability and Ethical Risks When LLMs Call External APIs

There is a moment when a language model stops generating text and starts generating consequences. The moment Tool Use in Prompts graduates from a research capability into production practice — when the model can query databases, submit forms, trigger workflows, initiate transactions — the ethical terrain shifts. We have built extensive frameworks for governing what AI says. We have barely begun to govern what AI does.

One Hundred Thousand Developers and the Server Nobody Approved

In August 2025, security researchers at Check Point disclosed CVE-2025-54136, a vulnerability they named MCPoison. The mechanism was direct: a malicious Model Context Protocol server could be swapped into a developer’s environment without triggering re-approval. The Tool Calling Schema layer — the structured interface through which LLMs register and invoke external capabilities — offered no protection against a substituted source. Affected: over 100,000 developers using Cursor IDE (Check Point Research). A companion vulnerability, CVE-2025-54135 (CurXecute), added remote code execution through prompt injection into auto-run MCP tools, with a CVSS score of 8.6 (Tenable).

That number — 100,000 — circulated quickly through security discourse, and rightly so. It is large enough to make the risk tangible. The patch arrived in Cursor version 1.3: MCP configuration changes now require mandatory user approval. The system corrected itself, as systems sometimes do.

Security note:

• CVE-2025-54136 (MCPoison): Cursor IDE MCP server swap without re-approval; over 100,000 developers affected. Disclosed August 2025.
• CVE-2025-54135 (CurXecute): CVSS 8.6 — prompt injection into auto-run MCP tools enabling remote code execution. Both patched in Cursor v1.3, released July 29, 2025. Fix: update to Cursor v1.3 or later.

Two Ways to Read an Exploit

The optimistic reading is tempting. Security researchers found the problem. Vendors disclosed it. A fix was released within weeks. This is exactly how responsible disclosure is supposed to work — and it worked. The number 100,000 becomes evidence of a functioning ecosystem rather than a failing one.

The critical reading begins where the patch ends. Cursor fixed the server-swap approval gap. What no patch can fix is the structural question the vulnerability surfaced: when an LLM calls an external tool on behalf of a user, whose judgment is being exercised? The model selected the tool. The model composed the parameters. The model decided that this action, at this moment, was the right response to this instruction. When Parallel Tool Calling patterns allow multiple actions to execute simultaneously before any human can review them, the approval window the patch restored may be narrower than it appears.

The question is not whether 100,000 developers were adequately protected after July 29, 2025. The question is who would have borne responsibility if they had not been.

The Assumption We All Share

Both readings assume that accountability exists somewhere in the chain. The optimistic version locates it in the vendor — who responded, who patched. The critical version places it with the deployer, or the developer, or the user who granted the LLM access to their tools. The disagreement is about where to point. The shared assumption is that there is somewhere to point.

OWASP LLM06:2025 names “Excessive Agency” as a distinct threat category — not a subcategory of Prompt Injection, but a risk class in its own right, driven by three root causes: excessive functionality, excessive permissions, and excessive autonomy (OWASP Gen AI). The taxonomy is useful. But taxonomies do not assign responsibility. They describe what went wrong, in the vocabulary of an industry that was built around systems where a human remained in the control path.

Accountability still lives somewhere in the chain — this is the invisible axiom that both the optimistic and critical readings inherit. We built our ethical and legal frameworks around it. What happens when it no longer holds?

When Accountability Becomes Mathematically Impossible

A 2026 paper by Tibebu and Shemtaga introduced what they call the Accountability Horizon — a formal impossibility theorem for governing human-agent collectives. Their argument is precise: above a threshold of autonomy, four properties of meaningful accountability (Attributability, Foreseeability Bound, Non-Vacuity, and Completeness) cannot be simultaneously satisfied. The theorem was validated across a large body of synthetic collectives with no exceptions found (Accountability Horizon paper).

The argument is not that accountability becomes difficult above a certain autonomy level. It is that accountability becomes structurally impossible — not because of poor engineering or absent governance, but because the mathematical conditions for it cannot be jointly maintained. You can satisfy three of the four properties. You cannot satisfy all four.

The draft NIST AI Risk Management Framework Agentic Profile, published March 2026, proposes four autonomy tiers — from full human supervision to full autonomy — with a requirement that deployers inventory tool risk across four dimensions: consequence scope, reversibility, authentication, and compositional risk (CSA Labs). This is careful governance design. But the NIST profile remains a draft whitepaper, not a finalized standard. And it does not resolve the impossibility theorem — it proposes controls within the portion of the autonomy spectrum where accountability is still tractable.

The EU AI Act places deployer accountability in Article 26: whoever deploys an AI system is accountable for its outputs regardless of who built it. The AI Liability Directive, which had proposed clearer fault mechanisms for AI-caused harm, was withdrawn in February 2025 — liability for AI-caused harm now routes through the revised Product Liability Directive (EU AI Act). Legal frameworks are being written in real time, on foundations not yet tested against the theorem.

What does it mean to hold a deployer accountable for a system whose outputs, above a threshold autonomy, cannot be fully attributed? The Structured Output Prompting layer enforces format; Constrained Decoding constrains tokens. Neither constrains consequences.

The Governance We Need Has Not Been Built

Thesis: Giving LLMs access to external APIs does not merely create security risk — it creates an accountability gap that existing legal, technical, and ethical frameworks are not designed to close.

The tools we use to reason about responsibility — causation, foreseeability, intent — were built for human agents making discrete decisions. They were extended, with some strain, to organizational actors and automated systems where a human remained visible in the loop. They were not built for systems that execute JSON Schema-defined tool calls at inference speed, in response to inputs that may have been manipulated through prompt injection vectors that no human observed in real time.

Prompt Testing And Evaluation can tell us how a system behaves under known conditions. Prompt Optimization reduces error rates. Libraries like Instructor enforce structured outputs at the integration layer. None of these practices create accountability — they create predictability within bounded conditions. The conditions under which an LLM calls external APIs are rarely bounded. Malicious MCP packages have circulated undetected for weeks before exposure — the harm they caused in those windows could not be attributed to any single decision.

What the Number Cannot Measure

The number 100,000 counts developers exposed to a documented vulnerability in a specific piece of software. It cannot count the decisions made, the data moved, the actions executed — undetected — before the exposure was understood.

It cannot measure the actions that leave no trace. Prompt injection through agentic pipelines does not always produce visible anomalies — it produces plausible outputs from a compromised instruction set. MCP, introduced by Anthropic in late 2024 (Anthropic Blog), has already undergone significant specification revision — evolving faster than the governance frameworks being built around it. Emerging governance frameworks for agentic AI consistently identify Explainability as the foundational pillar — the property that matters most for accountability, and the one least guaranteed by current architectures.

Anthropic’s model specification states that Claude must avoid “irreversible harm in the world” in agentic tasks (Anthropic’s Constitution). The intention is genuine. But which actions are reversible is unverifiable at the moment of execution — and the model is rarely in a position to know the difference. A database write can be rolled back. A message submitted to a third-party API cannot be unsubmitted. An authorization token exercised cannot be unexerted. The human, often, is not present to decide.

Where This Argument Is Weakest

This argument rests on the assumption that the autonomy trajectory continues upward — that agentic systems will acquire more tool access, not less, and that the accountability gap will widen rather than narrow. That assumption could be wrong.

Governance frameworks do respond to demonstrated risk. The NIST draft Agentic Profile represents serious institutional attention to exactly this problem. If deployer accountability under the EU AI Act produces meaningful litigation — if courts interpret Article 26 in ways that create real pressure to constrain tool scope and mandate human oversight — the incentive structure changes. The impossibility theorem may apply to current architectures at current autonomy levels; it does not necessarily apply to architectures explicitly designed around verifiable human checkpoints.

The Lifecycle Governance thinking emerging across agentic AI frameworks, and the practice of continuous evaluation as an audit function rather than a pre-release gate, suggest paths toward accountability that the theorem may not fully foreclose. The argument is also weakest when applied to narrow, high-stakes domains where tool access is genuinely constrained by design — medical AI with mandatory human review, for instance — rather than general-purpose agentic systems where tool scope is defined by whoever wrote the system message.

The Question That Remains

We have given language models the ability to act. We have not given our governance frameworks the ability to see what they did, attribute it cleanly, and hold someone responsible — not because governance is slow, but because above a certain autonomy threshold the attribution itself becomes mathematically unavailable. The accountability horizon is not a distant theoretical limit. It is the production environment that thousands of systems are already running in.

What does it mean to design an ethical AI system when the definition of “ethical” requires an accountability mechanism that the architecture, at scale, cannot provide?