Agent Memory and State: What Carries Between Steps

The Moment I Realized Agents Have a Memory Problem

When I first built a simple AI agent, I thought the hard part was getting it to take actions — calling tools, reading files, that kind of thing. Turns out the hard part is keeping track of what it already did.

My agent would run three or four steps, then suddenly "forget" what it had found in step one and repeat the same web search. Or worse, it'd carry so much accumulated context from previous steps that by step eight, the model was basically swimming in noise and making increasingly weird decisions. I'd hit the token ceiling way before the task was finished.

So I went back to basics. What does an agent actually need to remember? Where should that memory live? And how do you stop the context from ballooning into something unmanageable? That's what this article is about.

The Four Things an Agent Needs to Carry

Before talking about where to store things, it helps to be clear about what needs to be stored. In my experience, there are four main categories:

1. The original goal. What was the agent asked to do in the first place? This sounds obvious, but as steps accumulate, the original instruction can get buried under tool outputs, error messages, and intermediate results. The agent needs to keep this anchored somewhere prominent.

2. What's been done so far. A running log of completed steps. Not necessarily the full content of each step — just enough to know "I've already searched for X, read file Y, and written a draft to Z." This prevents redundant loops.

3. Key findings. The actual outputs that matter. If the agent searched the web and found three useful facts, those facts need to live somewhere the next step can use them — without dragging along the entire raw search response.

4. Temporary scratch space. Sometimes an agent is mid-calculation or mid-decision. It needs somewhere to hold half-formed thoughts before committing them to the next action. Think of it like a notepad on the desk.

Where Memory Actually Lives

Here's where things get concrete. There are three main places state can live in an agent system, and they each have different trade-offs.

In the context window (in-context memory). The simplest approach — just keep everything in the conversation. The agent's messages, tool results, and notes all pile up in one big thread. Easy to implement, but it has a hard ceiling. Once you hit the token limit, something has to give. Claude Code's auto-compact feature handles this gracefully, but you shouldn't rely on that as your only strategy.

In external storage (out-of-context memory). For anything longer-lived, you pull important state out of the conversation and store it somewhere else — a file, a database, even a simple JSON object on disk. The agent reads from it when needed and writes back when something important changes. This is how more robust agents handle large tasks.

In tool outputs (ephemeral memory). Sometimes the memory IS the tool result. If you call a "read file" tool and get back its contents, that output lives in the context for the next step to use. It's temporary — only as persistent as the current turn — but it's often all you need.

A Simple State Object Pattern

One pattern I keep coming back to is giving the agent an explicit state object at the start of each step. Instead of just dumping everything into a single long thread, you maintain a structured summary that gets updated and passed forward.

Here's a minimal example of what that looks like in practice:

At the end of each step, the agent updates this object — adds to steps_completed, appends relevant key_findings, and sets the next_action. The full raw tool output from that step can be discarded or summarized — what matters is the distilled insight that makes it into key_findings.

This pattern dramatically cuts down on context bloat. Instead of carrying fifteen paragraphs of raw search results forward, you carry three bullet points of what actually mattered.

How Context Balloons (And How to Stop It)

The most common mistake I see — and made myself — is treating the context window like a filing cabinet. Every tool call, every intermediate result, every error message just gets appended to the thread. By step six, the model is reading a novel just to answer the question "what should I do next?"

Here are three specific things that cause context to balloon unnecessarily:

Raw tool output dumping. When a tool returns 2,000 tokens of data and you pipe all of it into the next prompt, you've immediately burned a huge chunk of context on stuff that's probably 80% irrelevant. Instead, have the agent summarize the tool output before moving on. "The web search returned 10 results. The three most relevant were: ..." is vastly more efficient.

Full conversation replay. Some agent frameworks replay the entire message history at each step. This can feel safe — the agent always has "everything" — but it's a fast path to hitting the limit. Better to replay a structured summary, not raw transcripts.

Error accumulation. Errors pile up fast. If a tool fails three times, you don't need all three error messages in context forever. Keep the count, keep the final error state, and drop the redundant copies.

What to Keep vs. What to Drop

This is honestly the hardest judgment call in agent design. You don't want to drop something the agent will need later, but you also can't keep everything. Here's the mental model I use:

Keep forever: The original goal. Completed step names (not content). Final extracted facts. Any decisions the agent made and why.

Keep temporarily: Raw tool outputs — but only until you've extracted the relevant pieces from them. Error messages — until the error is resolved or retried.

Drop aggressively: Intermediate reasoning that led to a decision you've already made. Duplicate or near-duplicate tool results. Anything the agent already acted on and won't need to revisit.

In Claude Code specifically, when you're running a long agentic task, I'll often pause mid-session and manually ask Claude to "write a brief summary of what we've done so far and what's left." Then I use that summary as the anchor for the next phase, rather than letting the full history drag along. It's a manual version of what good agent systems do automatically.

Practical Checklist for Your Next Agent

When you're setting up an agent task — whether you're building one from scratch or just running a long Claude Code session — run through this quick checklist:

✅ Is the original goal written down somewhere prominent, not buried three hundred messages deep?

✅ Am I summarizing tool outputs before continuing, or dumping raw results?

✅ Do I have a way to track which steps are complete so the agent doesn't repeat them?

✅ Are important findings extracted into a separate, lean notes structure?

✅ Is there a plan for what happens when context gets too long — a compaction step, an external file, something?

If you can check all five, your agent is in good shape. If not, you've found your next thing to fix.

Memory management sounds like an advanced topic, but it's really just good housekeeping. The agents that work well aren't the ones with the biggest context windows — they're the ones that are ruthless about keeping only what actually matters. Once I started thinking about state like a minimalist packing list instead of a "bring everything just in case" bag, everything started working a lot better.