Prompt Injection Attacks in the Wild: AI Copilots, Email Agents, and AgentDojo Results

The security community warned about Prompt Injection for years. Production systems ran the experiment without them.

Microsoft 365 Copilot, Slack AI, and Bing Chat all broke. Real credentials. Real documents. Real data extracted from real users. The threat that looked theoretical in 2022 has a body count measured in exposed API keys and organizational files.

The defenses are now arriving. Whether they arrive before the next breach is the question.

The Attack That Keeps Repeating

Thesis: Every major production AI breach of the past two years was structurally the same attack — delivered through different surfaces, but rooted in the same architectural blind spot.

The vector is Indirect Prompt Injection. The attacker doesn’t talk to the AI directly. They plant malicious instructions inside external content — an email, a web page, a public channel — and wait for the AI to read it. The AI, trained to follow instructions in text, executes them. It cannot distinguish a legitimate user command from an attacker’s payload embedded in a retrieved document.

This is not a product defect. Any AI agent that reads external content and can take actions is structurally vulnerable until its architecture enforces Trust Boundary separation between what a user says and what the environment says.

Palo Alto Unit 42 documented 22 distinct payload engineering techniques observed in the wild as of March 2026 (Unit 42, Palo Alto): zero-sizing, off-screen positioning, CSS suppression, HTML attribute cloaking, runtime payload assembly. These aren’t experiments — the optimization phase has already begun. OWASP rates prompt injection as LLM01:2025, the top threat to large language model deployments (OWASP GenAI).

Three Breaches, One Blueprint

Bing Chat (2023) was the proof of concept that escaped the lab. Researcher Johann Rehberger demonstrated that maliciously crafted text on an external web page — rendered invisible via 0-point font — could inject commands into Bing Chat and extract email IDs and financial information from the user’s other open browser tabs (embracethered.com). Microsoft acknowledged the issue and deployed a content security policy patch restricting image loading to trusted domains.

Slack AI (2024) showed the same attack working inside enterprise infrastructure at scale. Security firm PromptArmor disclosed that injected instructions planted in public Slack channel messages could manipulate Slack AI into leaking private-channel content and API keys — exfiltrated and encoded into URLs — without the target user being present in the compromised channel. The attack surface expanded when Slack AI added DM files to its context window on August 14, 2024, the same day the vulnerability was disclosed (PromptArmor). Slack patched on August 21, 2024 (The Register).

EchoLeak (2025) raised the severity ceiling. CVE-2025-32711 (CVSS 9.3) required zero user interaction. A single crafted email was sufficient to extract chat logs, OneDrive files, SharePoint content, and Teams messages from Microsoft 365 Copilot. Researchers Pavan Reddy and Aditya Sanjay Gujral demonstrated the attack bypassed Microsoft’s cross-prompt injection attack classifier, link redaction, reference-style Markdown rendering, content security policy, and Teams proxy — all five defensive layers failed simultaneously (Hack The Box Blog). No confirmed malicious exploitation in the wild; Microsoft deployed emergency patches, though no specific patch date has been publicly confirmed (The Hacker News).

Security & compatibility notes:

• EchoLeak (CVE-2025-32711, CVSS 9.3): Zero-click indirect prompt injection in Microsoft 365 Copilot. Emergency patches deployed per reporting; patch date unconfirmed. No confirmed malicious exploitation in the wild per The Hacker News and SOC Prime.
• Slack AI: Indirect prompt injection patched August 21, 2024. The underlying injection class — instructions embedded in external content — remains active across AI agent deployments industry-wide.

Three systems. Three surfaces. One structural failure.

Where the Defenses Are Actually Working

Security researchers now have a rigorous benchmark: AgentDojo, presented at NeurIPS 2024 (NeurIPS 2024). It covers 97 realistic user tasks across 629 injection test cases in four environments: banking, Slack, travel, and workspace.

Under the original GPT-4o baseline, agent task completion dropped from 69% to 45% under attack. The canonical “Important message” injection achieved a 53.1% attack success rate on GPT-4o at baseline (EmergentMind, from NeurIPS 2024 baseline data).

Note: the benchmark has evolved since initial publication. More capable frontier models now approach near-zero attack success rates on the current version of AgentDojo without dedicated defenses — model-level improvements are compressing the problem, though not eliminating it (EmergentMind). Treat the original baseline numbers as a floor, not a current state.

Two defense approaches now carry peer-reviewed results:

MELON (ICML 2025) uses masked re-execution: it reruns the agent trajectory with the user prompt masked and flags tool calls that appear independent of the user’s actual task — and requires no model retraining whatsoever. MELON-Aug drops attack success to 0.32% on AgentDojo while preserving 68.72% of agent utility — inference-time only (arxiv, MELON paper).

PromptArmor — the defense paper submitted July 2025, distinct from the security company of the same name that disclosed the Slack vulnerability — uses an LLM preprocessor to detect and strip injected instructions before the agent processes external content. Results on AgentDojo using GPT-4o, GPT-4.1, and o4-mini: under 1% false positive rate, under 1% false negative rate, under 1% attack success (arxiv, PromptArmor paper). Treat as peer-reviewed preprint; venue unconfirmed.

Architectural defenses work at the design layer. Enforcing Privilege Separation between agent capabilities and external content processing limits blast radius before any attack lands. Constrained Decoding via JSON Schema enforcement — available through libraries such as Instructor, BAML, or XGrammar — eliminates entire classes of exfiltration by making it structurally impossible for an agent to produce arbitrary output. Structured Output Prompting further tightens the instruction surface attackers can target.

The research has answered whether defenses work. They do. The only open variable is how fast teams deploy them.

Who’s Still Running Without Cover

Every enterprise running an AI agent that reads external content and can take actions — email, documents, web, Slack.

Most security teams are scanning for SQL injection on a stack where the new attack surface is natural language embedded in every document the AI reads.

You’re either running injection red-team exercises on your AI agent deployments now, or you’re discovering the breach in a post-incident report.

What Happens Next

Base case (most likely): Inference-time defenses — MELON, LLM preprocessing, output constraining — get productized into major AI frameworks within 18 months. Most enterprises remain behind the curve during the deployment gap.
Signal to watch: Production-grade AgentDojo-successor benchmarks tracking defenses under adversarial adaptation.
Timeline: 2026-2027.

Bull case: Frontier model improvements continue compressing attack success rates without dedicated defense layers, making most injection attacks unreliable in well-configured deployments.
Signal: Consistent near-zero attack success in enterprise red-team exercises without MELON-class defenses.
Timeline: Late 2026 into 2027.

Bear case: Attackers adapt to masked re-execution detection. Novel attack channels — multimodal payloads, tool output poisoning, slow-burn agent memory manipulation — outpace current benchmark coverage.
Signal: Attack classes that defeat causal detection and LLM preprocessing simultaneously.
Timeline: Ongoing from 2026.

Frequently Asked Questions

Q: What are the most significant real-world prompt injection attacks on AI assistants and agents in 2024-2025?
A: Slack AI leaked API keys and private-channel content via indirect injection, patched August 2024. Microsoft 365 Copilot’s EchoLeak (CVE-2025-32711, CVSS 9.3) exposed chat logs and OneDrive files via a single crafted email, requiring zero user interaction.

Q: How did indirect prompt injection attacks compromise AI email agents and Microsoft Copilot?
A: Attackers embedded malicious instructions in emails or Slack messages. When the AI processed the external text, it executed the injected commands — bypassing multiple classifier layers in EchoLeak’s case, exfiltrating data encoded into URLs in Slack AI’s case, with no user action required.

Q: How did the Bing Chat prompt injection attack expose the indirect injection risk in production AI?
A: In 2023, researcher Johann Rehberger showed that invisible text on a malicious web page could inject commands into Bing Chat, extracting email IDs and financial data from other open browser tabs. Microsoft patched via content security policy restrictions after acknowledging the issue.

Q: How are MELON, PromptArmor, and the AgentDojo benchmark shaping prompt injection defenses in 2026?
A: AgentDojo (NeurIPS 2024) established the first realistic multi-environment benchmark for injection defenses. MELON (ICML 2025) drives attack success below 1% without model retraining. The PromptArmor paper achieves under 1% on all three key error dimensions using an LLM preprocessor.

The Bottom Line

Prompt injection moved from academic threat to production incident in under two years. Three major systems broke. The defenses now exist and work at benchmark scale. The gap is production deployment — and that gap is your current risk exposure.