Context Failure Modes: How Long Conversations Quietly Go Wrong

The Conversation That Ate Itself

I was about forty messages deep into a Claude session last month, refactoring a messy Express API. Things had been going great. Then I asked a pretty simple question — something like "can you add input validation to the user route?" — and got back code that used a completely different library than the one we'd been using for the past hour. Not wrong, exactly. Just... drifted. It was answering a version of my question that existed in some imagined conversation, not the actual one we'd been having.

That was my wake-up call. Long AI conversations don't just run out of steam the way you might expect. They don't slow down or start saying "I don't know." They quietly change shape. The outputs look confident. They're just subtly wrong in ways that are easy to miss if you're not paying attention.

There are three main failure modes I've identified — what I call poisoning, distraction, and drift. Each one degrades your conversation in a different way, shows up at different moments, and has different tells. Let's go through all three.

Failure Mode #1: Context Poisoning

Poisoning is the most dangerous one because it's often your own fault — and you introduced the problem early, which means it infects everything that comes after.

Here's how it happens. You're getting started on a project and you paste in some context — maybe a code snippet, a design doc, some notes you jotted down. But that context has a mistake in it. Maybe you described the wrong database schema, or you had an outdated assumption about how your API works. You didn't notice at the time. The AI didn't flag it. And now every response is built on that faulty foundation.

The sneaky part is that models are trained to be helpful and consistent. They're not going to suddenly contradict the context you gave them fifteen messages ago. They'll just... keep building on the bad information. Your scaffolding looks solid from the outside. The foundation is cracked.

The best defense is verification checkpoints. After you dump a big block of context at the start of a session, ask the model to summarize back what it understands about your setup. Make it explicit:

If the summary has errors, you catch them before they replicate through a hundred messages of work.

Failure Mode #2: Context Distraction

Distraction is what happens when the conversation gets too long and the model starts paying more attention to the wrong parts of it.

Here's the thing about transformer-based models and attention: not all tokens in your context window are weighted equally. As your conversation grows, the model has to split its attention across more and more content. Recent messages tend to get more weight. Important things you said twenty exchanges ago? They fade. And if you went on a tangent — even a useful one — that tangent can start drowning out the stuff that matters.

I ran into this building a custom CLI tool. We had a long detour where I asked the AI to help me understand a library I ended up not using. Twenty messages later, the AI was still referencing that library in its suggestions, even though we'd explicitly abandoned it. The tangent had more recency weight than the decision to move on.

The distraction tell is subtler than poisoning. You'll notice the AI starts being less precise about your actual requirements, or it hedges more, or it reaches for generic solutions instead of the specific approach you established earlier. It's not wrong — it's just not paying attention to the right things.

The practical fix is what I call a "ground truth" message. Periodically drop in a message that re-states the things that matter most — even if you've said them before:

It feels redundant. It works. The model's attention snaps back to what you actually care about.

Failure Mode #3: Context Drift

Drift is the most insidious one because it happens slowly, and it feels like progress the whole time.

Drift is what happens when the conversation itself changes your goals. You start working on one thing. Someone — maybe you, maybe the AI — suggests a slightly different framing. You go with it. Then another small adjustment. And another. By message forty-five, you're technically still working on "the app," but what you're building looks nothing like what you set out to create, and you can't point to any single moment where things went sideways.

I've watched this happen across weeks on a side project. I started building a simple habit tracker. Three weeks of AI-assisted development later, I had a gamified productivity system with a points economy and achievement badges. Every single step felt logical in context. The cumulative drift was enormous.

The drift tell: if you read your latest exchange and then your first few messages and feel a vague sense of "wait, how did we get here?" — that's drift. The output is coherent within its own bubble. It's just not the output you originally needed.

The fix for drift is a written north star — a single sentence or paragraph you write at the very start of a session that defines exactly what success looks like. Not vague. Specific. Keep it visible. Use it as a measuring stick:

When you catch yourself adding a feature the AI suggested that wasn't in that definition — pause. It might be a good idea. It also might be drift. Decide consciously, not by momentum.

A Quick Diagnostic You Can Run Right Now

If you're in a long session and something feels off but you can't name it, try this:

The answer tells you which failure mode you're in. Missing constraints = distraction. Contradictory details = poisoning. A goal that doesn't match your original intent = drift.

These three failure modes — poisoning, distraction, and drift — aren't bugs you can patch. They're structural properties of how large language models work with long contexts. The context window isn't like RAM that stores everything equally. It's more like human working memory: limited, recency-biased, and easily hijacked by the loudest or most recent thing in the room.

The good news is that once you know the names for what's happening, you catch it fast. I've started treating long AI sessions the way I treat long coding sessions — with explicit checkpoints, written goals, and a healthy skepticism when something feels too smooth to be real. Your AI isn't trying to mislead you. It's just doing its best with a context that quietly got away from both of you.