Vendor Lock-In, Data Sovereignty, and the Ethics of Routing All AI Traffic Through a Single Chokepoint

Every day, thousands of engineering teams route their AI inference through managed LLM Gateway layers — proxy services that sit between their applications and the model APIs they call. The abstraction is genuinely useful: one interface, multiple providers, unified observability, centralized Rate Limiting and cost controls. The value is real. But abstractions that become single points of passage carry a specific kind of risk, one that is structurally invisible until something collapses through them.

The LiteLLM incident brought that structure into sharp relief. And it arrived alongside something equally consequential — an acquisition wave that, within a single quarter of 2026, removed two of the four most-cited independent gateways from the category of independent products altogether.

One Dependency, Three Vulnerability Classes

The March 24th supply chain attack was not an isolated event in LiteLLM’s 2026 security record. In the weeks that followed, researchers disclosed three further vulnerabilities: a SQL injection flaw in the proxy’s API key verification carrying a CVSS score of 9.3, exploited within 36 hours of public disclosure (The Hacker News); an MCP injection flaw added to the CISA Known Exploited Vulnerabilities catalog in June 2026 with active in-the-wild exploitation confirmed (CSA Labs); and an authorization bypass chain allowing a low-privilege user to escalate to full administrative access and remote code execution (The Hacker News). Three distinct vulnerability classes, across four separate disclosures, within three months.

OWASP classifies this cluster of risks under LLM Supply Chain (LLM03), Sensitive Information Disclosure (LLM02), and Data Poisoning (LLM04) — not as edge cases, but as expected attack surfaces for systems with this architecture (OWASP Gen AI). What is less often discussed is what made these vulnerabilities so consequential in practice. It was not the code alone. It was the architecture. When you route all your AI traffic through a single API Gateway, you inherit every flaw in that layer — not just across your applications, but across your credential stores, your model routing logic, and the runtime context you assumed was isolated from external interference.

Centralized routing is centralized exposure.

Security & compatibility notes:

• LiteLLM Supply Chain (March 24, 2026): Versions 1.82.7 & 1.82.8 contained credential-harvesting malware (~119k downloads, ~40-minute window). Advisory GHSA-5mg7-485q-xm76; targeted AWS/GCP/Azure credentials, SSH keys, Kubernetes tokens (LiteLLM Security Blog, PyPI Blog).
• LiteLLM SQL Injection (CVSS 9.3): CVE-2026-42208 in proxy API key verification; fixed in v1.83.7-stable (Apr 19, 2026). Exploited within 36 hours of public disclosure (The Hacker News).
• LiteLLM MCP Injection (CVE-2026-42271): Added to CISA Known Exploited Vulnerabilities catalog June 8, 2026; active in-the-wild exploitation confirmed. Source: CSA Labs.
• LiteLLM Authorization Bypass: CVE-2026-47101 — low-privilege user escalation to full admin and remote code execution, published June 15, 2026. Verify severity against official LiteLLM security advisory before acting (The Hacker News).

The Profile You Didn’t Know You Were Building

There is a quieter question underneath the supply chain incident, one that the security disclosure process tends to obscure: the gateway that was compromised saw everything before it was compromised. Prompts, responses, latency, cost-per-call, model selection patterns, request volume by time of day. This is the observability value proposition. It is also, structurally, a comprehensive operational profile of your AI systems that lives outside your infrastructure.

Virtual Keys and scoped credential management can limit the blast radius of certain credential failures. They do not address the aggregate picture the gateway accumulates over time. Who owns that picture? Under what conditions can it be used for purposes beyond routing your calls? Most vendor onboarding flows do not answer these questions — because the terms of engagement were written by the party with the aggregate view.

Braintrust records every LLM call as a structured trace — prompt, response, cost, latency — and offers SOC 2 Type II, HIPAA, and GDPR compliance with VPC and air-gapped deployment options. This is a meaningfully more defensible data posture than most alternatives, and the Fallback Strategy management it provides is genuinely useful. But Braintrust’s Data Processing Addendum contains a distinction worth reading carefully: for customer personal data, Braintrust acts as a processor — the customer remains the controller. For “Account Data” and “Usage Data,” however, Braintrust acts as an independent controller, governing that data category by its own privacy policy rather than the customer’s processing instructions (Braintrust’s DPA). The boundary between “Account Data” and the operational contents of interaction logs is not defined in a way that makes the distinction obvious from the outside.

Even the more defensible data postures contain a structural risk that compliance certifications do not resolve: deletion requests may not remove data from backups or training datasets, and no standardized format exists for exporting conversation history across LLM gateway vendors (nezhar.com). The absence of portability is not a design oversight. It is how dependency deepens, silently, over time.

The Case for the Chokepoint

The defense of managed gateways deserves more than a dismissal, and the forensic arc demands we take it seriously.

Building your own routing layer — managing provider outages, model fallover, per-model rate limits, cost allocation across teams — is genuinely hard at any organizational scale. The breakeven for self-hosted infrastructure to become cost-competitive sits at roughly 100 to 200 million tokens per month (nezhar.com). Below that threshold, the economics make the case against managed gateways mostly theoretical. And the alternative to a managed gateway is not clean autonomy: it is multiple different SDK integrations, multiple credential stores, and multiple failure modes, all without unified observability. The abstraction reduces friction. For teams below a certain scale, it genuinely reduces risk by consolidating what would otherwise be fragmented exposure.

The compliance argument carries weight too. SOC 2 attestations, regional hosting in EU and US jurisdictions, OWASP-aligned security protocols — these exist in the category, and organizations operating under GDPR or healthcare data requirements have found configurations that satisfy their legal obligations. The case for the chokepoint, at its best, is a case about what complexity the gateway absorbs so your team does not have to. This argument is not wrong. That is precisely why what happened next matters.

What the Acquisitions Changed

The defense rests on one assumption that 2026 has made difficult to sustain: that the organization you chose will remain the organization running your data.

Helicone, an open-source gateway with Apache 2.0 licensing and a documented commitment to data autonomy, was acquired by Mintlify on March 3, 2026, and entered maintenance mode — patches only, no new features (ChatForest Review). What the maintenance mode announcement did not surface: Helicone’s privacy policy designates it as Data Controller, not a data processor — a classification that places direct regulatory responsibility for how that data is used on Helicone itself, not on the customer (Helicone’s privacy policy). Retention is documented as “as long as necessary,” with no specific periods, and the policy contains no clause addressing whether request and response logs might serve any purpose beyond the immediate service. Nor does it rule out such use.

Portkey, which was processing trillions of tokens per month at the time of its acquisition, was absorbed by Palo Alto Networks on May 29, 2026 and is now a component of Prisma AIRS, a security platform (Palo Alto Networks). The post-acquisition data handling policies for Portkey are not yet publicly documented.

Two of the four most-cited independent gateway products ceased to exist as independent products in the same quarter. Teams that chose these gateways based on their privacy postures, their organizational culture, their commitment to open-source principles — they did not choose their acquirers. Their due diligence was rendered retroactively incomplete by a transaction they had no visibility into and no power to prevent.

This is not an argument against gateways. It is an observation about what “independent” means when independence can be dissolved in a press release, without guaranteed data portability, without clarity about how historical traffic will be handled going forward, and without any mechanism for the customer to object.

Inference Without Accountability

Thesis: Routing AI inference through third-party gateways is a legitimate architectural choice, but normalizing it without governance obligations leaves organizations exposed to risks that no compliance certification resolves.

The question of who benefits from the aggregate view is rarely framed in terms of accountability, because accountability implies someone answerable to someone else. At least 34 countries have enacted or strengthened data localization laws restricting where AI processing can occur (AI Magicx). EU AI Act Article 10, which requires data governance documentation for high-risk AI systems, enters full enforcement in August 2026 — a deadline that was already on the calendar when most of these gateway contracts were signed (EU AI Act Service Desk). An Austrian regulator fined a fintech €450,000 for using a U.S.-based AI API — a number that represents not the ceiling of regulatory pressure but an early indicator of its direction (AI Magicx). The regulatory trajectory is already visible, even if the gap between regulatory intent and enforcement remains wide.

What has not arrived is an accountability layer specific to AI inference routing: mandatory portability guarantees, regulated data handling disclosures, incident notification timelines, audit trails for closed systems. The 93% of executives who identify AI sovereignty as mission-critical have presumably internalized the exposure (Kai Waehner). The institutional mechanisms to address it have not caught up. In their absence, each organization is writing its own rules for an infrastructure category that has not yet decided what rules it should follow — and the organizations with the most to lose are often the ones least equipped to write those rules well.

Where This Argument Is Weakest

A fair skeptic would note that the market is already differentiating. Braintrust’s air-gapped deployment option exists. LiteLLM’s self-hosted path is viable for teams at sufficient token volume. Helicone’s Apache 2.0 license offers a path to ownership for those willing to operate it. The instruments of sovereignty are available; the critique is that most teams don’t use them, because convenience and organizational inertia route traffic toward the managed default.

That is a correct observation, and it narrows the scope of the argument I am making. I am not claiming that defensible architectures are unavailable. I am arguing that when the default path — managed, hosted, convenient — carries no mandatory accountability, traffic concentrates there. And when infrastructure critical to AI inference concentrates in layers with no guaranteed portability, no regulated data handling disclosures, and no mechanism for customers to audit what happens to their data during an acquisition, the exposure is not individual. It is systemic.

The acquisitions of 2026 did not create this condition. They made it legible.

The Question That Remains

We have built, quietly and without much deliberation, a routing layer for some of the most sensitive inference workloads in modern software — and we have placed that layer with a small number of vendors whose long-term ownership, data practices, and accountability to their users are not guaranteed by design. The right question is not whether this was intentional. The right question is whether we would have chosen it if it had been made explicit.

What does it mean to trust an infrastructure layer that can be sold to a security conglomerate on a Thursday, with no right of exit written into the contract you signed?