Your agent forgot what you told it three turns ago, so you reached for the obvious fix: a model with a bigger context window. Stuff the whole conversation in. Stuff the whole codebase in. A million tokens, surely that is enough to remember everything. Then the agent got worse. It started looping on its own earlier guesses, picking the wrong tool out of a list it could see perfectly well, confidently citing a fact that was sitting right there in the prompt, wrong. You did not run out of memory. You poisoned it.
I have been running a cross-session memory layer for Claude for months now, Context Hub, the thing that decides what a model gets to see at the start of every session. I built it because I believed the same thing you probably believe: that the job of a memory system is to remember more. That was the wrong mental model, and it cost me weeks of agents that felt lobotomized despite having access to everything. The reframe that fixed it is the opposite of the instinct: the job of a memory layer is not to remember. It is to forget, aggressively, on purpose, and to forget the right things.
Here is the model I wish someone had handed me, with the research that proves it is not just my anecdote.
The myth: context is storage
The instinct treats the context window like RAM you fill up: more capacity, more retained, better performance. That model is wrong in a specific, measurable way, and the measurement now exists.
Chroma published a study in July 2025 called Context Rot that tested 18 frontier models, GPT-4.1, Claude 4, Gemini 2.5, Qwen3, the whole roster, on tasks where they deliberately held the difficulty constant and only changed one thing: how much surrounding text the model had to wade through. Same question, same single answer hidden in the haystack, just more irrelevant tokens around it. If context were storage, accuracy would stay flat. It did not. Performance degraded as input length grew, on every single model, even though the task never got harder.
The number that should rearrange your priorities: degradation showed up as early as 500 words in some tasks. Not 500 thousand. Five hundred. In the simplest test they ran, copying a sequence of repeated words with one oddball inserted, models started failing as the sequence got longer, sometimes repeating words that were never there, sometimes refusing, sometimes emitting noise. A task a regex could pass, failing because the input got long. The “lost in the middle” effect compounds it: models attend faithfully to the start and end of a context and let the middle blur, so the bigger the window, the more of your information lands in the dead zone.
So the 1M-token number on the spec sheet is not a promise. It is a ceiling on what you can cram, not a guarantee of what the model can use. Treat it like a disk you can fill and it rots.
The proof that hurts: less context, better answers
The finding from that study that genuinely changed how I build: Chroma ran a long-conversation benchmark where the model could answer either from the full multi-turn history or from a short condensed summary that preserved the same facts. The full version contains strictly more information. If more context helps, the full version wins, always.
It lost. For some models, accuracy dropped by around 30 percentage points when fed the full conversation instead of the condensed one, despite the full version containing every fact the short one did, plus more. The extra context was not neutral padding. It was active interference.
Sit with that, because it inverts the whole intuition. The condensed context did not win because it was complete. It won because it was clean. Every irrelevant turn you leave in the window is a distractor competing for the model’s attention against the one thing that matters. Adding context that is not load-bearing does not add safety. It adds noise, and noise is negative.
This is why the discipline has a new name. A year ago the bottleneck was prompt engineering, how you phrase the instruction. Now it is context engineering: deciding what the model knows, sees, and remembers at the exact moment it acts. Datadog’s 2026 telemetry across thousands of real agent traces found that 69% of input tokens are already system prompts and tool instructions, and concluded flatly that context quality, not context size, is the limiting factor for agent reliability. Most teams never come close to filling their window. They are losing on what they put in it, not how much fits.
What this means for the thing you are building
If you maintain anything that decides what an LLM sees, a RAG pipeline, an agent loop, a memory layer, a chat app that appends history, you are a context engineer whether you signed up for it or not. The reframe gives you a different set of questions to ask of your own system.
Retrieval is a precision problem, not a recall problem. The naive RAG move is to grab the top 50 chunks and dump them in, on the theory that more candidates means the answer is probably in there somewhere. It is in there, drowning. The chunk that answers the question is now one of fifty, forty-nine of which are semantically similar enough to compete for attention and wrong. Retrieve wide if you must, but then re-rank hard and feed the model a small, high-confidence slice. The answer being present in the context is not the goal. The answer being findable is.
Long agent trajectories need active pruning, not accumulation. The default agent loop appends every observation, every tool result, every intermediate thought to the context and carries it forward forever. By turn 30 the agent is reasoning over a transcript thick with its own abandoned guesses, and it cannot tell which of its past statements were dead ends and which were conclusions. The research backs the brutal fix: in code-editing agents, simply masking old observations, or summarizing them away, cut cost by more than half with no loss in solve rate, and in several setups the pruned agent beat the one that kept everything. The most recent context is usually sufficient. The full history is usually a liability.
Treat every token you add as a cost, not a freebie. This is the mental flip. The old question was “what might be useful to include?” and the answer to that is always “more,” which is how you rot the context. The new question is “what earns its place in the window right now?” Default to exclusion. Make each piece of context justify itself against the noise it adds. A memory layer that returns ten relevant things and zero irrelevant ones beats one that returns the same ten buried in forty.
That is the rule I run Context Hub on now. It does not try to surface everything it knows about a session. It tries to surface the fewest things that let the model act correctly, and it treats every extra item it includes as a debt against the model’s attention. The version that remembered more was worse than the version that forgets well. I had to build the greedy one first to believe that.
The reality check
This is not a license to starve your model. Forget the wrong things and you get an agent that is confidently ignorant, which is its own failure mode. The skill is not minimization, it is curation: maximize the relevant, minimize everything else, and accept that “everything else” includes a lot of stuff that feels safe to keep. There is no clean threshold I can hand you, no “keep it under N tokens” rule that holds across tasks, because the cliff depends on the model, the task, and how semantically close your distractors are to your answer. You have to measure it on your own workload. Build the eval that feeds your system the full context and the condensed context and compares, because until you have watched less context win on your own data, the instinct to hoard will keep winning the argument.
But the direction is settled. The teams whose agents hold up in 2026 are not the ones with the biggest windows. They are the ones with the most disciplined forgetting.
I am rebuilding Context Hub’s retrieval around this, ranking aggressively and returning less, and I will write up what the eval actually showed once I have the numbers, including where forgetting the wrong thing bit me. If you maintain a memory or RAG layer, I want to know: have you measured the point where adding context started hurting on your own workload, or are we all still assuming more is safe? Tell me what you found.