Context windows, memory, and state.

Why memory is where most agent failures begin CORE

The loop from Lesson 1.3 carries state across turns — everything the agent has learned, decided, or read so far. State is the agent’s working memory for the task. Without it, every turn of the loop would start from nothing.

But state does not live in some magical infinite space. It lives inside the model’s context window — a fixed-size buffer of tokens the model can see on any given turn. When the state grows beyond what the window can hold, something has to leave. And when something leaves, the model behaves as if that information never existed.

Memory is easy to skim past in the mental model — which is exactly why it is where most beginners get tripped up. Students who can describe how a model reasons will still hit memory failures if they do not understand how context, scratchpads, and external state relate. When an agent's output goes sideways, the cause is more often something missing from what the model can see than something the model itself got wrong. This lesson makes that surface visible so you can direct around it.

Three layers of agent memory CORE

Agents have access to memory at three different time scales. They are not interchangeable.

Short-term — the context window. This is the tokens the model can actually see on the current turn. It includes the original task, the agent’s prior reasoning, prior tool calls and results, and any system instructions. Different models have different window sizes; as of this writing, windows range from about 8,000 tokens on smaller models to 200,000+ on the largest. A rough conversion: 1,000 tokens ≈ 750 English words. To make those numbers feel real: an 8,000-token window is roughly 6 single-spaced pages of text — enough for one short essay and a handful of follow-up questions. A 200,000-token window is roughly a 600-page novel — enough that you'd have to be working pretty hard to fill it. When you start a new conversation, the short-term window resets to empty.

Medium-term — in-task scratchpads. During one task, the agent can write notes to itself: a file saying “sources found so far,” a short running summary, a plan it updates as it goes. These notes survive inside the task but only if the agent deliberately writes them and reads them back. There is nothing automatic about medium-term memory — it happens because the agent (or the director) designed the task to produce and consume notes.

Long-term — external state. Files on disk, databases, calendars, email archives, note apps, a dedicated memory file. Anything that lives outside this conversation and can be read by a tool call next week. Long-term memory survives across tasks and sessions. It is the only kind of memory that persists after the context window is wiped.

The student’s most common wrong model is “the AI remembers me.” It does not, except where you have given it a tool and directed it to look. Every piece of information the agent will use on the next turn, you must either keep in the context window or make retrievable via a tool.

Why context windows fill up, and what happens when they do CORE

Everything in the conversation costs tokens. The original task costs tokens. Each tool result costs tokens — sometimes a lot of tokens, if the tool returned a 40-page PDF. The model’s own reasoning text costs tokens. A long back-and-forth conversation with an agent can easily consume tens of thousands of tokens even without attachments.

When state grows toward the cap, one of two things happens:

1. The system rejects the request. Common when you try to upload a huge file or paste a very long document. You get an error. This is the friendly failure mode, because at least it tells you.
2. The system silently truncates. Usually by dropping the oldest content from the window. The model stops seeing what happened earlier in the conversation, but its new turns still produce confident output based on what is left in the window. There is typically no alarm, no warning, and no visible sign in the trace that the window just lost its memory of Turn 3.

The second failure mode is the dangerous one. The agent appears to be working; it just happens to be working from an incomplete picture. You notice the problem when it contradicts something it agreed to ten minutes ago, repeats a step it already did, or loses track of a constraint you stated up front. That moment of “why is it suddenly acting like it forgot?” is almost always a context-window truncation at work.

Four “forgetting” modes, and the directing move for each CORE

When an agent seems to forget something, it is always one of four distinct modes. Naming the mode tells you the fix.

1. Context-window truncation. The conversation grew past the window; the oldest tokens were dropped. Fix: use a model with a larger window, shorten the up-front context, or ask the agent to write a running summary to a scratchpad file.
2. New conversation reset. You started a new chat, and the window is empty. The agent has no memory of the prior session — even if “you” are the same user, the tokens are gone. Fix: put the information the agent needs into external memory (a file, a memory system, a document) and direct the agent to read it at the start of the new conversation.
3. Tool result never made it into state. The agent called a tool, but the result was too large, was filtered, or never came back. The next turn sees no evidence it happened. Fix: check the trace (Lesson 1.3), find the missing result, and rework the task so the tool call returns something small enough to fit or gets summarized before adding to state.
4. External state not consulted. The information lives in a file the agent could read, but on this task the agent did not look. Fix: be explicit in the task — “first read notes.md, then proceed” — or build a habit into the agent’s instructions so it checks relevant external state every time.

The skill is the diagnosis. “The agent forgot something” is uninformative. “We hit the window cap and lost the third turn” tells you exactly what to do next.

Three directing moves for memory CORE

Three moves turn up over and over as you advance through the course.

Budget context. Give the agent what it needs to start, not your whole hard drive. A sharply-scoped task with 2,000 tokens of relevant input beats a vague task with 20,000 tokens of loosely-related input almost every time. When you are tempted to paste “just in case” material, first ask whether the agent actually needs it for this turn. If it does not, leave it out.

Use external memory for anything that must persist. If the answer needs to survive past this conversation — or past this turn, for a really long task — write it somewhere. A file. A memory system. A document. Do not rely on the context window to hold something important across a session boundary. The context window is working memory; it is not long-term storage.

Split long tasks. A task that needs 300,000 tokens of reasoning is almost always better structured as three tasks of 100,000 tokens, each producing a file that the next task reads. You trade a little orchestration for enormous gains in reliability and cost. Module 8 is entirely about this pattern; you are meeting the core intuition here.

When an agent seems to forget something, ask: which of the four forgetting modes, and which of the three directing moves fixes it? Two questions, and you'll have a working diagnosis for nearly any memory problem you meet.