Context Rot, Lost-in-the-Middle, and the Hard Technical Limits of Multi-Turn LLM Conversations

At conversation turn fifteen, the LLM contradicts a constraint it accepted at turn three. Not because the context window overflowed — the session sits comfortably within the advertised limit. Not because the model reconsidered. Something else has failed: quietly, predictably, and architecturally. The mechanisms responsible operate in parallel from the first exchange, and no amount of additional context capacity fully neutralizes them.

Multi-turn conversations expose at least three overlapping failure modes. They interact. And they are not edge cases — as of mid-2026, every major frontier model exhibits them.

Why Attention Has a Structural Memory Problem

The Context Window is not a neutral container. Every generation pass computes attention scores between the current token and every token in context, then normalizes those scores through softmax — a function that forces weights to sum to one across the entire context. That normalization is fundamental to the mechanism.

And it has a consequence that is easy to miss.

Softmax attention is zero-sum. A conversation with two hundred tokens distributes attention across a different probability landscape than one with twenty thousand — even when the same sentence occupies the same relative position in both. As context accumulates, each individual token receives proportionally less attention weight. Not as a semantic judgment. As an arithmetic consequence of the denominator growing.

Why do LLMs lose track of information from earlier in a long conversation?

Two mechanisms collaborate to make early-turn information fade.

The first is attention dilution. Each new turn appends tokens to the context. The softmax denominator grows. Early tokens — those from the first few conversation turns — receive decreasing attention weight relative to the current generation position, simply because more tokens now compete for a fixed allocation that always sums to one. The information is still there. It is proportionally quieter.

The second is positional encoding. Most production models use Rotary Position Embedding (RoPE), which encodes relative position as a rotation matrix applied to token embeddings. RoPE enforces long-term decay by design: tokens far apart in sequence have their attention scores naturally reduced by the angular distance between their position vectors. A token at position 1 and one at position 18,000 interact with structurally weaker attention than those same tokens at positions 1 and 50 — regardless of how semantically important the early token is.

These two mechanisms compound in every multi-turn session. The System Prompts definition established at conversation initialization moves progressively further from the current generation position in sequence space — and progressively lower in attention weight — as each subsequent exchange appends more tokens. The role definition, output constraints, and behavioral rules placed at the start of a session are subject to the same structural decay as any other early-context content.

The model has not forgotten the system prompt. The attention pathway to it has narrowed.

The U-Shaped Curve: Position Bias in Long Contexts

The second failure mode operates even when the context is not especially large. Liu et al. 2023 (Lost in the Middle, Transactions of the Association for Computational Linguistics) studied how model performance on multi-document question answering shifts as a function of document position within the context. The results traced a U-shape: accuracy ran at 70–75% when the relevant document was first or last in a sequence of twenty documents, and dropped to 55–60% when it sat in the middle — a fifteen to twenty percentage-point penalty determined entirely by position, not by content quality (Liu et al. 2023).

The relevant document did not change. Its location did. That was sufficient to predict the model’s accuracy.

What is the lost-in-the-middle problem in multi-turn LLM conversations?

The lost-in-the-middle problem is this: models trained on next-token prediction develop disproportionate attention toward tokens at the beginning and end of their context, with middle positions systematically underweighted — regardless of whether those middle positions contain the most relevant information.

In a multi-turn conversation, this translates directly. Instructions placed early enjoy primacy bias — briefly. As subsequent turns accumulate, early instructions shift into the middle-context range, where they enter the accuracy trough. Recent turns crowd the recency zone and receive disproportionate weight. The model’s output reflects this geometry: a constraint from turn twelve is structurally more influential than an equally precise constraint from turn three, not because of semantic content, but because of position in the attention distribution.

Content relevance cannot override positional geometry.

Hsieh et al. 2024 (Found in the Middle, ACL Findings) confirmed the U-shaped bias mechanistically, attributing it partly to RoPE’s long-term decay properties and to residual connections that create a recency anchor in the model’s final layers (Hsieh et al. 2024). Their calibration method — adjusting attention weights to correct for positional bias — improved retrieval-augmented generation performance by up to fifteen percentage points. That is a useful measure of how much position bias costs in practice. It is also a measure of how much headroom remains unrealized in conversations structured without explicit attention to positional distribution.

Notice what this implies for Prompt Engineering strategy: the location of an instruction inside the conversation history is not merely a formatting concern. It is a performance variable — one that shifts with every subsequent exchange.

When Context Grows: The Degradation Threshold

The third mechanism becomes relevant when conversations extend to a significant fraction of the model’s stated context capacity — though “significant fraction” arrives earlier than most users expect.

Wang et al. 2026 characterized an “intelligence degradation” threshold: the point at which task performance drops more than thirty percent from baseline. For Qwen2.5-7B, that threshold arrived at 40–50% of maximum context length, where F1 scores fell from 0.55–0.56 to approximately 0.3 — a 45.5% degradation (Wang et al. 2026). The behavior was cliff-like rather than gradual. The threshold is specific to that model and architecture; generalizing exact percentages to larger models requires caution. But the cliff pattern — a non-linear drop rather than a smooth slope — appears consistent across the architectures studied.

Context rot begins before the context window fills.

How does context window overflow degrade multi-turn conversation quality?

A model with a 200,000-token context window can exhibit measurable degradation at 50,000 tokens — well before the advertised boundary (Morph 2026). Most frontier models perform reliably at approximately 60–70% of their stated maximum; beyond that range, attention dilution and positional bias compound into performance characteristics that diverge substantially from single-turn behavior (Elvex 2026).

At the behavioral level, Laban et al. 2025 measured the aggregate cost across more than 200,000 simulated conversations spanning six generation tasks and major open and closed models: an average 39% performance drop moving from single-turn to multi-turn interactions, alongside a 112% increase in output unreliability (Laban et al. 2025). The thirty-nine percent figure is an average across diverse task types and models; individual variance is substantial, and the number should not be read as applying uniformly to any specific model or scenario.

Dongre et al. 2026 offers the most mechanistically precise account of what is actually failing. Their “Goal Accessibility Ratio” (GAR) tracks how accessible goal-defining tokens remain through the attention mechanism as turns accumulate. For Mistral, GAR fell from near-perfect on short conversations to approximately eleven percent on a twenty-fact retention task (Dongre et al. 2026, May 2026 preprint). The same study then applied linear probes to the model’s residual representations and found the missing information encoded with 0.99 AUC accuracy. The model had the facts. They were present in the weights. The attention pathway to them had closed.

This distinction is the clearest description of what context rot means architecturally: not data loss, but progressive inaccessibility. As of mid-2026, all eighteen frontier models tested in the Morph analysis — including GPT-4.1, Claude Opus 4, and Gemini 2.5 Pro — showed measurable performance degradation as context grew (Morph 2026). Context window size determines when the degradation begins. It does not prevent it.

What the Architecture Predicts About Your Conversations

These three mechanisms — attention dilution, position bias, and the cliff-like degradation threshold — are not independent failure modes. They interact, and they generate testable predictions.

If a critical constraint is defined in the Context Engineering layer at session initialization, expect its structural influence to decline as turns accumulate. RoPE decay and softmax dilution act simultaneously; each new turn widens the attention gap for prior content. This is especially consequential for Instruction Following scenarios where the model must reference an early constraint while responding to a recent query — the structural weighting favors the recent, regardless of which carries more semantic authority.

If you insert mid-conversation re-statements of key instructions, you counteract position bias but consume context budget — pushing the conversation closer to the degradation threshold. The two mitigations trade against each other.

If the model contradicts a commitment established earlier in a session, the failure is most likely positional rather than reasoning. The model has not reconsidered. It has generated output that reflects the current context’s attention distribution, in which the earlier commitment is structurally underweighted. No reasoning error occurred. The constraint was simply outweighed by geometry.

Constraints designed to prevent Prompt Leakage are equally subject to attention decay — system-prompt instructions that anchor information-disclosure boundaries fade at the same structural rate as any other early-context content, which has direct implications for long-session security.

Research suggests that instruction-following accuracy degrades measurably over short conversations as well; one figure often cited places the drop at seventeen percentage points between turn one and turn three for a leading model, though this comes from summary sources rather than direct paper abstracts and should be treated as indicative, not definitive.

Rule of thumb: treat any constraint placed more than five turns ago as structurally weakened, regardless of whether the context window appears to have available capacity.

When it breaks: context rot is most severe in tasks requiring long-range coherence — iterative Structured Output generation, extended Role Prompting sessions where the persona was defined at turn one, and Meta Prompting pipelines that append context across multiple reasoning stages. In these scenarios, the model’s failures are not errors of reasoning. They are the predictable output of an attention mechanism functioning exactly as designed, in a context structure the mechanism was not built to maintain coherently over time.

The Data Says

Three overlapping mechanisms degrade multi-turn LLM conversations: softmax attention dilution reduces the structural influence of early tokens as context grows; the U-shaped position bias systematically disadvantages middle-context information regardless of content quality; and a non-linear degradation threshold causes cliff-like performance drops that arrive well before the advertised context window fills. Expanding the context window shifts where these effects begin. It does not resolve any of them.