Why Long Conversations Go Sideways
Here's something that used to drive me absolutely crazy when I first started using ChatGPT seriously. I'd start a conversation, ask a few questions, get some great answers — and then 20 messages in, the AI would start giving me these weirdly vague, hedging, off-topic responses. I'd be thinking: "We were just talking about this. What happened?"
What happened is that the context got messy. Every message you send — and every response the AI generates — eats into the model's available context window. And as that window fills up with old messages, earlier instructions, and back-and-forth chatter, the AI's "attention" gets diluted. It's trying to hold too much at once, and the useful signal gets buried in noise.
I call the fix for this the "Shrinking Context" technique. It's not a single prompt trick — it's a small system for resetting, summarizing, and refocusing your AI as a conversation grows. Once I started using it, the quality of my long working sessions with AI jumped noticeably.
What's Actually Happening Inside a Long Chat
Before we get into the technique, it helps to understand the mechanics. AI models like GPT-4 or Claude have what's called a context window — basically a limit on how much text they can "see" at once. Every message in your conversation thread counts toward that limit.
Modern models have large context windows (sometimes 128k tokens or more), but size isn't the whole story. Research and practical experience both show that models tend to pay more attention to the beginning and end of a conversation, with the stuff in the middle getting less "weight." So if your original system prompt or key instructions are buried under 40 messages, the AI might not be holding onto them as firmly as you think.
This is why you'll notice drift in long conversations — the AI starts contradicting earlier decisions, forgetting constraints you gave it, or getting generically helpful instead of specifically useful.
The Middle-of-Conversation Dead Zone
If you gave the AI important constraints or context somewhere around message 10 out of 30, there's a real chance it's not weighting that information heavily anymore. Don't assume the AI remembers — remind it.
The Three-Part Shrinking Context System
The technique breaks down into three moves you can use at different points in a long conversation. You don't have to use all three every time — sometimes just one is enough. Think of them as tools in your belt.
1. The Rolling Summary Request
Every 10–15 messages, ask the AI to summarize where you are. This does two things: it surfaces whether the AI actually still has a grip on the project, and it gives you a compact snapshot you can paste into a fresh conversation if needed.
# Rolling Summary Request
Before we continue, give me a 5-bullet summary of:
→ What we've decided so far
→ What constraints are in place
→ What the next open question is
Keep it tight. I'll use this to reorient us.2. The Context Injection Reset
When you feel the conversation drifting, don't just ask your next question. Start your message with a quick re-anchoring block. I do this all the time when I'm working on a coding project across a long session.
# Context Injection Reset
Quick re-anchor before my next question:
→ Project: Building a Flask API for a task manager
→ Stack: Python 3.11, Flask, SQLite
→ Constraint: No external auth libraries, keep it simple
→ Last decision: We're using UUIDs for task IDs
With that in mind, here's my next question: [your question]3. The Fresh Start with Memory Paste
Sometimes the conversation is just too far gone and the cleanest move is to start a new chat. But instead of starting from scratch, you paste in the summary from step one as your opening message. I think of this as "carrying your notes to a new whiteboard."
# Fresh Start with Memory Paste
Continuing a project from a previous session. Here's the context:
→ [Paste your 5-bullet summary here]
Pick up from there. My next question is: [your question]Signals That Tell You It's Time to Reset
You don't need to reset on a fixed schedule — you just need to recognize the warning signs. Here's what I watch for:
The AI contradicts an earlier decision. If you decided on a specific approach in message 5 and the AI is now suggesting something different without acknowledging the change, context drift is happening.
Answers get vague and hedgy. Instead of specific, confident responses, you start getting "it depends" and "there are several approaches you could take." The AI is compensating for uncertainty about what you actually want.
The AI stops using your terminology. When I'm working on a project, I use specific names for things. If the AI starts using generic terms instead of the names we established together, it's losing the plot.
You have to re-explain something you already covered. This one's obvious but easy to miss in the moment. If the AI asks a clarifying question about something you addressed 10 messages ago, reset time.
Build the Habit Early
Don't wait until the conversation is broken to start summarizing. I now do a rolling summary request proactively around message 12–15, before I notice any drift. It takes 30 seconds and saves way more time than it costs.
A Real Example: How I Used This on a Writing Project
A few months back I was using Claude to help me outline and draft a longer tutorial series. We were about 25 messages in — we'd agreed on a specific tone, a particular reader persona, a structure for each article, and a list of topics to cover in a specific order.
Around message 22, I noticed Claude suggesting an outline structure we'd explicitly decided against earlier. The old me would have just gone along with it or gotten frustrated. Instead, I did a quick context injection reset:
# Re-anchoring mid-session
Hold on — let me re-anchor us before we go further.
→ Audience: Career changers learning AI tools, not developers
→ Tone: Conversational, first person, like a friend explaining
→ Structure: Problem → Why it matters → The technique → Example → Tips
→ We decided NOT to use numbered step lists as the primary format
Please re-draft that outline with this structure in mind.The next response was dramatically better. Claude course-corrected immediately, and we finished the session without any more drift. That little reset probably saved me 30 minutes of back-and-forth.
Making This a Repeatable Habit
The hardest part of this technique isn't learning it — it's remembering to use it when you're deep in a flow state and things seem like they're going fine. Here's how I've made it stick:
I keep a small text file called ai-session-notes.md open whenever I'm doing a long AI work session. Every time I hit a key decision or constraint, I jot it down in bullet form. This becomes my "memory paste" material if I need to do a fresh start.
I also set a loose mental rule: if I'm scrolling up to re-read old messages to figure out what we decided, that's my cue to do a rolling summary request before continuing. Scrolling up to re-read is a symptom — the summary is the cure.
Finally, don't feel bad about starting fresh conversations. There's no prize for keeping a single chat thread going forever. A clean new conversation with a well-crafted opening context block will almost always outperform a 40-message thread that's gotten muddy. The AI doesn't have feelings about it — you're not abandoning anyone.
Save Your Best Summaries
When you generate a rolling summary that really nails the project context, save it. That's a reusable asset — you can use it to start any future session on the same project without rebuilding context from scratch.
Long conversations with AI are genuinely powerful — you can build up a lot of shared context and tackle complex, multi-step problems in ways that just aren't possible in a single quick prompt. But that power only holds if you actively maintain the quality of that context. The Shrinking Context technique is how you do that. It takes maybe an extra minute here and there, and it's absolutely worth it.
More tutorials in this category, or explore the full field guide.