The Moment I Realized Something Was Missing
About six months into using AI seriously, I had a long debugging session with Claude. We'd been going back and forth for maybe forty-five minutes — pasting code, discussing errors, refining the approach. Then I asked a follow-up question that referenced something we'd established near the start. The response came back completely wrong. Not a little wrong. Like we'd never discussed it at all.
I assumed the AI had gotten confused or I'd phrased something badly. I rephrased. Same result. I eventually scrolled up and realized: that context was probably gone. The model literally could not see it anymore.
That was my first real encounter with the context window — and it changed how I use AI tools completely.
What the Context Window Actually Is
Think of the context window as the model's working memory. It's not long-term storage. It's not a database. It's a fixed-size whiteboard that holds everything the model can see right now: your system prompt, every message in the conversation, any documents you've pasted in, and every response the model has generated.
When that whiteboard fills up, something has to go. And here's the part that catches most people off guard: the model doesn't warn you. It doesn't say "heads up, I can no longer see your first message." It just... silently drops the oldest content and keeps going as if nothing happened.
Context windows are measured in tokens, not words or characters. A token is roughly 3-4 characters of English text, so one word averages about 1.3 tokens. A single page of normal prose is around 500 tokens. Code tends to be token-heavier because of all the symbols and formatting.
Here's a rough sense of current context window sizes across popular models:
# Approximate context windows (mid-2025)
GPT-4o → 128,000 tokens (~96,000 words)
Claude 3.5 Sonnet → 200,000 tokens (~150,000 words)
Gemini 1.5 Pro → 1,000,000 tokens (~750,000 words)
Llama 3.1 70B → 128,000 tokens (~96,000 words)
# Note: bigger ≠ better attention across full windowThose numbers sound enormous. And they are, until you start pasting in a full codebase, a long document, and a conversation history. Then they shrink fast.
A Real Conversation Hitting the Limit
Let me walk you through what this actually looks like in practice. Say you're building a Flask app and you paste your entire app.py into the chat — 400 lines of code. That's maybe 8,000 tokens right there.
Then the conversation unfolds:
# Turn 1 — You paste 400 lines of app.py (~8,000 tokens)
You: "Here's my Flask app. Can you help me add rate limiting?"
AI: "Sure! Here's how to add Flask-Limiter..." (~600 tokens)
# Turn 2 — follow-up, including AI's full previous response in context
You: "Great, now let's add JWT authentication." (~20 tokens)
AI: "For JWT auth you'll want flask-jwt-extended..." (~800 tokens)
# Turn 3 — more back and forth, context growing
You: "Now let's refactor the database layer." (~15 tokens)
AI: Long response with refactored code examples... (~2,000 tokens)
# Turn 15 — context window is filling up
# Original app.py is now being dropped from the window
You: "Wait, does this still work with my original database setup?"
AI: "I don't have context about your original database setup..."
# ↑ This is the moment. The model genuinely can't see it anymore.The model isn't being difficult or forgetful in some human sense. The original code literally isn't on the whiteboard anymore. It got erased to make room for everything that came after it.
What Falls Out First (And Why It Matters)
Most models use a simple rule: when the context fills, the oldest messages get dropped first. This means your carefully crafted initial context — the setup, the constraints, the background information you provided at the start — is typically the first to go.
Your system prompt is usually protected (the model won't drop that), but everything else is fair game. This is particularly brutal for:
Long documents you pasted early on. The model read them and responded, but once they're out of the window, they're gone. Any questions referencing them will get vague or wrong answers.
Initial constraints you set. "Always use TypeScript, never JavaScript" might have been in message one. By message thirty, that constraint may no longer be visible to the model.
Code that's been revised multiple times. The model might be working from an intermediate version rather than the final one you thought you established.
The Silent Failure Problem
The dangerous thing isn't that the model fails obviously — it's that it often keeps answering confidently with whatever partial context it still has. You might not realize something important dropped out until you've gone several more turns in the wrong direction.
How to Check Your Context Usage
The honest answer is: it depends on the tool you're using, and most of them don't make it easy.
Claude.ai: There's no real-time token counter in the UI, but Claude will tell you directly if you ask. You can type "How full is your context window right now?" and it'll give you an honest estimate. Not perfect, but it works.
ChatGPT: The web interface doesn't show token counts. If you're using the API, every response includes usage.prompt_tokens and usage.completion_tokens in the response object, which is genuinely useful.
API usage (any provider): This is where you actually get numbers:
# OpenAI API response includes usage stats
response = client.chat.completions.create(...)
print(response.usage.prompt_tokens) # tokens you sent
print(response.usage.completion_tokens) # tokens in the response
print(response.usage.total_tokens) # both combined
# Quick manual estimate: paste text into
# platform.openai.com/tokenizer to count tokensFor quick estimates without writing code, OpenAI's tokenizer tool at platform.openai.com/tokenizer lets you paste any text and see the token count. Anthropic has a similar tool. These are genuinely useful for getting a sense of how "heavy" your prompts are before you send them.
What Context Actually Costs You
Here's where the context window becomes a financial question, not just a technical one. Every token in your context window costs money when you're using the API.
The pricing model is typically: you pay for input tokens (everything in the context window, including history) plus output tokens (the response). This means a long conversation gets progressively more expensive with every turn, because the growing history is being sent and processed each time.
# Rough cost example — GPT-4o at ~$2.50/1M input tokens
Turn 1: 8,000 tokens input → $0.02
Turn 5: 20,000 tokens input → $0.05
Turn 15: 60,000 tokens input → $0.15
Turn 25: 100,000 tokens input → $0.25
# Not huge per conversation, but it adds up fast
# at scale or in automated pipelinesFor casual chat use this is negligible. For production applications making hundreds or thousands of API calls, context management becomes a real cost-optimization problem. Teams spend significant engineering effort on techniques like context compression, summarization, and selective retrieval (which is basically what RAG is about) specifically to keep context lean.
Practical Strategies for Working Within Limits
Once you understand the constraint, you can work with it instead of fighting it.
Start a fresh conversation for new topics. This sounds obvious but it's the most effective thing you can do. Long conversations with topic drift are the fastest path to lost context.
Re-paste critical context when the conversation gets long. If there's a piece of code or a set of constraints that needs to stay visible, paste it again near the bottom of the conversation every handful of turns. It costs tokens, but it's much cheaper than going down the wrong path.
Summarize as you go. You can literally ask the AI: "Summarize the key decisions we've made so far in this conversation" — then copy that summary and start a new thread with it as the opening context. This is a technique I use constantly for long coding sessions.
Front-load the important stuff. Put your most critical constraints and context at the very beginning of the conversation, not buried in the middle. The beginning tends to be what matters most for setting tone and direction, even if it gets dropped later.
The "Context Checkpoint" Habit
Every 10-15 turns in a long session, I pause and ask: "What's your current understanding of what we're building and the constraints we're working within?" If the answer is off, I know the context has drifted and it's time to reset — before I waste another ten turns going the wrong direction.
Keep your inputs lean. Don't paste a 2,000-line file if only 200 lines are relevant. Don't include your full conversation history if you can link back to key decisions. Every unnecessary token is context you could be spending on something useful.
The Bigger Picture
Understanding the context window fundamentally changes how you interact with AI tools. Instead of treating a conversation like a relationship that builds indefinitely, you start thinking of it as a sliding window — always moving forward, always dropping the oldest content.
That mental model helps you stop being surprised when the AI "forgets" something you told it three hours ago, and start designing your conversations to keep the most important context visible at all times.
It's one of those things that seems like a technical limitation but is actually just a different way of working — and once you adapt to it, you get dramatically better results.
More tutorials in this category, or explore the full field guide.