TL;DR
- Context rot is performance degradation that happens as input context grows, well before the model hits its token limit. It affects every frontier model tested.
- The context limit is not the actual performance limit. Models start becoming unreliable at roughly 60-70% of their advertised window, with accuracy dropping 30%+ from position effects alone.
- Three mechanisms drive it: lost-in-the-middle (U-shaped attention), attention dilution (finite focus spread across tokens), and distractor interference (topically similar noise compounds the problem).
- A bigger window does not solve this. Context engineering does: compaction, instruction re-injection, sub-agent delegation, and treating the context budget as something you spend deliberately.
Your coding agent starts strong. First fifteen turns, it nails every instruction, catches every edge case, follows your conventions like it wrote them. Then somewhere around turn thirty it reintroduces a bug you fixed twenty minutes ago. It restates a constraint you gave it at the start. Its suggestions get vaguer, less specific, more generic. Nothing crashed. No tool call failed. The trace looks healthy.
That is context rot.
The term was formalized by Chroma's 2025 research, which tested 18 frontier models and found that every single one degrades as input length increases. Not some. Not most. All of them. And the degradation starts long before the context window fills up.
What Context Rot Actually Is
Context rot describes a specific, measurable phenomenon: as you add tokens to an LLM's input, the quality of its output decreases. This is not context window overflow. Overflow happens when you exceed the model's maximum token limit and messages get truncated. Rot happens well before that.
A model with a 200K token window can exhibit significant degradation at 50K tokens. The decline is continuous, not a cliff. Teams routinely assume their context window is big enough. They pick a model with 128K or 1M tokens, load it up, and wonder why output quality slides. Context rot explains why.
The distinction matters because the engineering response is different. Overflow is a hard stop you can detect and handle with truncation logic. Rot is a gradual slide that standard monitoring misses entirely. The trace looks healthy. The token count rises smoothly. But the model's grip on any one detail weakens as everything else piles in.
The Three Mechanisms
Context rot is not caused by a single bug. It emerges from three distinct failure modes, each with different causes.
Lost-in-the-Middle
Transformer attention favors the start and end of a prompt over the interior. This is the most mechanically documented mechanism. Stanford researchers placed a correct answer at different positions across a long context and measured retrieval accuracy:
- Beginning: approximately 75% accuracy
- End: approximately 72% accuracy
- Middle: approximately 55% accuracy
That is a 20 percentage point drop caused entirely by where the information sat, not by its quality or relevance. The information was present. The model could technically see it. It simply was not attending to it properly.
This creates a U-shaped attention curve that persists across model sizes, architectures, and training approaches. Even models explicitly designed for long-context processing exhibit it. The practical consequence: when your agent reads 8 files looking for a function, the relevant code in file four sits in the model's blind spot. It has the right information in its context but cannot effectively attend to it.
Attention Dilution
As the prompt grows, the model's attention budget spreads across more tokens. Instructions that were sharp at turn two compete with everything added since. Effective grip on any one detail weakens even though nothing was removed.
Transformer attention is O(n^2) with respect to sequence length. At 10K tokens, the model manages 100 million pairwise relationships. At 100K tokens, that jumps to 10 billion. Memory is not the bottleneck. Attention precision is. Each individual relationship gets proportionally less focus.
This is why context rot gets worse with scale. A 200K token window does not mean you should use 200K tokens. Signal-to-noise ratio determines output quality, not raw capacity.
Distractor Interference
Distractors are topically related to the information you care about, but do not quite answer the question. They are different from irrelevant content, which is unrelated. Chroma's research found that even a single distractor reduces performance relative to a baseline with no distractors, and four distractors compound the degradation further.
Different distractors have non-uniform impact. Some cause greater performance decline than others. And this non-uniformity amplifies as input length grows. In Chroma's experiments, certain distractors appeared most frequently in hallucinated responses across models, meaning the model was actively confused by similar-sounding information sitting nearby.
This is particularly dangerous in RAG pipelines. When you retrieve 10 document chunks per query and several of them are topically similar, the model has to disambiguate among them while the context window fills with more chunks. The distractor interference compounds with attention dilution.
How to Measure It
Context rot is hard to detect because the symptoms look like model capability problems. The agent gets dumber, so you assume the model is bad. But the model was fine at turn five. The context is what changed.
Production Signatures
The production signature of context rot is a gap that widens as turn count rises, with no sudden token drop. If prompt tokens rise smoothly across a session and quality falls after a turn-count threshold, that is context rot, not overflow.
Look for these signals:
- Agent forgetting instructions from earlier in the session
- Quality degrading after a specific turn count rather than at a specific token count
- Performance varying by context length on equivalent tasks (10x variance is documented)
- Agent time spent retrieving context climbing toward 60% of total processing
The 35-Minute Wall
Analysis of long-running agents found a critical threshold: every AI agent's success rate decreases after 35 minutes of human-equivalent task time. The relationship is non-linear. Doubling task duration quadruples the failure rate.
Why 35 minutes? That is roughly when context accumulation crosses a critical threshold for most coding tasks. By 35 minutes, the agent has typically read 15 to 30 files, run multiple searches, and accumulated 80K to 150K tokens of context. Even with a 200K token window, the signal-to-noise ratio has degraded enough that reasoning quality drops measurably.
This creates a compounding problem. As the agent gets less accurate, it makes mistakes. Mistakes require corrections. Corrections require reading more files and running more searches. The context grows faster as the agent gets worse.
The Advertised vs. Effective Window Gap
Models typically break 30 to 40 percent before their claimed limit. A 200K model becomes unreliable around 130K tokens. Degradation is often sudden, sharp drops rather than smooth decline. About two-thirds of tested models fail to find a simple sentence in only 2K tokens of relevant content buried in a larger context.
The practical implication: design systems to target 60 to 70 percent of advertised context as the working maximum. For a 1M token model, plan for 600K to 700K tokens of reliable content.
The Real-World Cost
Context rot is not an academic curiosity. It hits production systems hard.
Coding Agents
For coding agents, context rot is the primary failure mode. Not model capability. Not reasoning ability. The models are smart enough to solve the problem if their context stays clean. The problem is that context does not stay clean.
Agents accumulate noise during search, exploration, and backtracking. Each file read, each grep result, each dead end adds tokens. By the time the agent finds the right file, buried in the middle of 20,000 tokens of accumulated context, attention is spread thin. The analysis of the one file that actually matters is noticeably weaker than it would have been with a clean context.
This is why a coding agent reintroduces a bug you fixed twenty minutes ago. The fix is technically still in the transcript. It is just buried under the last forty tool calls, and buried is functionally gone.
Enterprise Failures
Nearly 65% of enterprise AI failures in 2025 were attributed to context drift or memory loss during multi-step reasoning. The area where agents fail or frustrate users the most is in the quality of their responses: 61% of respondents in the 2026 AI Engineering Report say hallucinations are the top failure mode. And context rot directly contributes to hallucination rates by degrading the model's ability to ground its responses in the provided context.
The financial impact compounds. You are paying for tokens that make your results worse. Users who are not happy with responses ask similar questions in different ways, driving up token costs while quality continues to slide.
RAG Pipelines
A retrieval pipeline that fetches 10 document chunks per query works fine for most queries. But longer queries trigger larger system prompts and conversation history. Total context grows to 40,000 tokens, and the documents retrieved third and fourth, sitting in the middle, fall into the attention dead zone. The model answers confidently, drawing on what it can see well. A crucial nuance from chunk four gets missed.
I have personally worked with RAG systems that retrieved 50 relevant chunks but performed worse than systems that retrieved only 5. More context is not automatically better.
Defenses Against Context Rot
The fix is not a bigger window. It is treating context as a finite attention budget you spend deliberately. Here are the defenses that actually work.
1. Context Compaction
Run compaction proactively, not just at capacity limits. Summarize older conversation turns. Drop tool outputs that are no longer relevant. Use compact diffs: never echo an entire file back into context when ten lines changed.
Properly implemented compression typically achieves 60 to 80 percent cost reduction without quality loss. The key word is "properly." Aggressive compression that drops details requiring re-fetching can exceed the token savings.
2. Instruction Re-Injection
Critical constraints get pushed toward the edge of the context window by everything that follows. The fix is deliberate repetition. Place your most important instructions at the beginning of the system prompt and repeat them near the actual task. This positions them in the high-attention zones.
Do not just state a constraint once in the middle. Repeat it at the beginning AND near the task. The constraint mentioned once in the middle is the constraint the model will forget.
3. Sub-Agent Delegation
Sub-agents provide context isolation by executing potentially noisy operations (file searches, code analysis, summarization) in separate context windows. Rather than polluting the parent agent's context with thousands of lines of grep and read outputs, sub-agents perform these operations independently and return only compact, structured results.
Meanwhile, the parent agent's context stays clean. A sub-agent absorbs the context hit on its isolated session. When the task is done, the main agent gets a clean summary. The main session stays limited and manageable.
This pattern works because each sub-agent approaches the task fresh, without being anchored by what other operations found. It is the same reason code reviews from multiple perspectives produce better results than a single reviewer accumulating context.
4. Context Isolation
Beyond sub-agents, isolate context at the architectural level. Do not load everything the system knows for every decision. Load only the context required for the detected intent and nothing else. Pricing logic does not mix with support responses. Onboarding flows do not interfere with sales.
Each decision happens inside a controlled environment. Intelligence is modular, not monolithic. Instructions are scoped, not shared.
5. Strategic Positioning
Place the most important information at the beginning and end of the context, where models typically perform best. This works with the U-shaped attention curve rather than against it. Put your system prompt first, the user's task last, and minimize what sits in the middle.
6. Active Context Pruning
Intelligent systems must manage context proactively, not reactively. Remove semantically redundant content before it reaches the model. Every token saved is attention capacity preserved. Use token counting libraries to accurately measure what you are sending.
7. Two-Stage Retrieval
Broad recall via vector search followed by precise cross-encoder reranking improves relevance by 15 to 30 percent compared to embedding-based retrieval alone. The reranker analyzes query-document pairs at inference time with full context, whereas embeddings compress documents into fixed vectors before queries arrive.
8. Sliding Windows
Keep recent context while dropping older parts. For long conversations, maintain a summary of older turns rather than the full transcript. The model gets the gist without the accumulated noise.
9. Hierarchical Memory
Move beyond temporal divisions to functional categories: factual memory (knowledge and facts), experiential memory (insights and learned skills), and working memory (active context management). Recent systems achieve 85 to 93 percent reduction in token usage compared to baseline methods.
10. Task-Specific Window Sizing
Size context budgets by task type, not a single percentage rule. Retrieval tasks tolerate larger windows. Synthesis and multi-document reasoning degrade much faster. Match your context strategy to the task.
11. Monitoring for Rot
Set up monitoring that correlates context length with downstream quality metrics. Track task success rates and user satisfaction against turn count, not just token count. The gap that widens as turn count rises, with no sudden token drop, is the production signature of context rot.
12. Externalizing Intent
Stop treating a long-running chat as your product's memory. The moment your intent matters, and it matters the instant the app is more than a throwaway, get it out of the transcript and into a record the agent reloads: a spec, acceptance criteria, a written definition of what done means. The builders who stay ahead of rot are not the ones with the biggest window. They are the ones whose source of truth was never in the window to begin with.
The Engineering Discipline
Context rot is one of those concepts that feels advanced until it is encountered in production, and then it feels like something that should have been understood from day one.
The core reality is simple: transformer attention is a finite, dilutable resource. Every token added to a context window reduces the focus available for everything else. When contexts grow long and important information ends up in the middle, quality degrades in ways that are real, measurable, and unfortunately silent.
The good news is that it is manageable. Compact early. Isolate exploration into subagents. Be precise with retrieval. Position critical content deliberately. None of these requires advanced machine learning knowledge. They are engineering disciplines applied to a new kind of resource.
Context rot is not going away. Windows will get bigger, attention will still be finite, and the sessions where you actually build real things will always run long enough to rot. The question is not whether your context will degrade. It is whether you have the engineering discipline to manage it before the silent failure shows up in your production traces.
Sources
- Chroma Technical Report: Context Rot — 18-model study documenting universal performance degradation as input length increases
- Liu et al., "Lost in the Middle" (Stanford, 2023) — Foundational research on U-shaped attention curves and positional bias in long contexts
- TheContextCompany: What is context rot in AI agents? — Technical breakdown of mechanisms and monitoring gaps
- MorphLLM: Context Rot Complete Guide — Analysis of the 35-minute threshold and practical defense patterns