Reviewing what the agent did.

Why this is the most important lesson in Module 3 CORE

You can get by without knowing the details of how the Code tab renders its panels or where the Claude Code CLI prints its status line. Tools change. What does not change — will not change for a long time — is the skill of reviewing the agent's work.

A change the agent makes is a claim. The claim is: “If you accept this, your program will do what you asked and nothing else.” Your job as the director is to check that claim. Not because agents are sloppy — frontier coding agents are good, often better than a human at writing the obvious-looking fix — but because “does what you asked” is not a property the agent can verify alone. Only you know what you asked for, and only you can confirm the result matches.

This lesson does three things. It names the four ways an agent's edit can go wrong. It gives you a written five-move review you can run on any change. And it puts that review on real changes — including one that looks right and is wrong, which is the category that catches everybody eventually.

Read this lesson slowly. The payoff is not in the next ten directed edits. It is in the next thousand.

Reviewing for a director, not a programmer CORE

You are not a programmer reading code. You are a director reviewing the agent's work. Those are different jobs and they call for different tools.

A programmer reviews a change by reading every line of code in the diff and judging whether each line is correct. That works if you can read the language fluently. This course does not require you to.

A director reviews a change by checking the shape of what the agent did, reading the agent's summary of the change, asking the agent to explain anything unfamiliar, running the result with realistic input, and iterating if anything is wrong. That works whether or not you can read the language — and it scales to every kind of agentic work, not just code.

One small bit of mechanics first: a diff is the set of changes between what a file used to contain and what it contains now. Lines removed appear with a - prefix (red); lines added appear with a + prefix (green); unchanged lines for context appear plain. You don't need to read the lines to use the diff — you just need to see the shape: which files, how big, what's added, what's removed, anything you didn't expect.

The five-move review is the rest of this lesson. Before that, the four things you're looking for.

The four failure modes CORE

Every agent change that goes wrong goes wrong in one of these four ways. Naming them gives you faster recognition when you hit one — and each one has a director's move that catches it without code-reading.

Misplaced change. The fix works here but won't work for the next similar case. Example: in Lesson 3.2 you'll direct the agent to fix a bug where certain filenames end up in the wrong folder — specifically, files with two dots in the name like notes.tar.gz. The agent could "patch" the bug by adding a one-off rule just for .tar.gz files, and the test would pass. But the next file with two dots in its name (a different kind of backup file, a different kind of archive) would have the exact same bug all over again, because the rule was glued onto one specific case rather than fixed at the root. The right fix lives in the part of the program that handles filenames in general, so it covers every similar case at once. How a director catches this: ask the agent — “Will this fix work for similar cases I haven't shown you, or only for the exact one I asked about?” A good agent will admit when its fix is narrow.

Scope creep. The agent does more than you asked. Example: you asked for a bug fix in tidy.py; the change also reformats three other files “for consistency.” The extras may be harmless, they may be helpful, and they may include a silent behavior change you did not ask for and will not notice until something breaks in two weeks. The rule is: unrequested changes are bugs until proven otherwise. How a director catches this: look at the shape of the change. Ask the agent — “Did you change anything I didn't ask for?” Ask it directly to list every file it touched and why.

Silent deletion. The agent removes something that was carrying real behavior, and the deletion isn't replaced. Example: a line that creates a folder before writing to it gets removed because “the system handles that automatically” — which isn't quite true, and the program now crashes the first time someone runs it on a fresh machine. The behavior that was there yesterday is gone, and the agent's summary may not mention it. How a director catches this: ask the agent — “What did you delete and why?” And run the result against realistic inputs that exercise behavior the change might have removed.

Plausible wrong. The change looks right, runs without errors, passes the tests — but does the wrong thing on a case the tests didn't cover. Example: the file-sorting script from Lesson 3.2 has tests for files like report.pdf, photo.jpg, and notes.txt. The agent's fix passes all of them. But the first time you run the script on your real Downloads folder, it crashes on a file with no extension at all — a screenshot saved with just a name, or a download the website didn't label. The tests pass because nobody wrote a test for "file with no extension." This is the hardest failure mode to catch, and it's the one that most rewards running the result against real inputs (your actual Downloads folder, your actual notes, your actual data) rather than just trusting that "tests pass" means "works."

Naming these four failure modes gives you four questions you can ask of any diff — the four diff-review questions the rest of the course refers back to:

1. Right place? — does the change land where it should, in a way that will generalize? (Catches misplaced change.)
2. Right scope? — did the agent change only what you asked? (Catches scope creep.)
3. Legitimate deletion? — if anything was removed, did the agent name it and replace its behavior? (Catches silent deletion.)
4. Any surprise? — tests pass and the diff looks reasonable, but does the result match the goal on real input? (Catches plausible wrong.)

These four questions are the compressed form of the failure-mode taxonomy. The five-move review below is how you actually run them on a diff.

The five-move review CORE

Here is how a director runs the review. Five moves, in order, every time. The first time it will feel slow. By the tenth time it will compress into a single pass.

Move 1 — Check the shape. Before anything else, look at what files changed and how big the change is. Most agent tools show this at the top of the diff: “3 files changed, 14 lines added, 6 lines removed.” Ask yourself:

• Did the agent change the files I expected, and only those files?
• Is the size of the change roughly what I expected? (A one-line bug fix shouldn't be 80 lines.)
• Are there any new files I didn't ask for, or files deleted that I didn't ask to remove?

If the answer to any of these is “no” or “I'm not sure,” that's the cue to slow down. Shape is the cheapest and fastest signal you have — most scope-creep failures are visible from the file list alone, before you read a single line of the change.

Move 2 — Read the agent's summary. Every modern coding agent gives you a plain-English summary of what it did. The Code tab does this inline next to the diff; the Claude Code CLI does it in the terminal. Read it. Then compare it, sentence by sentence, against the prompt you wrote.

• Does the summary describe what you asked for?
• Does it mention things you did not ask for? (Those are scope creep.)
• Does it mention removing something? (That deserves a follow-up — see Move 3.)
• If the summary is vague (“made the changes”, “fixed the issue”), ask the agent to be more specific: “List every change you made and why.”

The summary is the agent's claim about what it did. Move 1 checks the shape of the work; Move 2 checks the story of the work.

Move 3 — Ask the agent about anything unfamiliar. This is the move most non-coders skip and shouldn't. For any line, file, or concept in the change that you can't tell about, ask the agent in plain English:

• “What does this part do?”
• “Why did you make this change?”
• “What happens if I run this on an empty input? On a really large input? On a file with weird characters?”
• “Are there any cases where this fix won't work?”
• “Did you remove anything that was doing real work?”

The agent already has full context on the change — it just made it. Asking is the fastest diagnostic you have. If the agent's answer doesn't satisfy you, that is itself information: the change isn't ready to accept yet. This is the move that scales — every kind of work the rest of this course teaches uses the same pattern. When uncertain, ask the agent.

Move 4 — Run the result. Tests passing is not the same as “the change works.” A test exercises one case the test author thought to write. Real input is the hundred cases nobody thought of.

For coding work specifically: actually run the script on a realistic input and confirm the output matches what you wanted. Not what the test wanted — what you wanted. If the change is supposed to fix a bug, reproduce the bug yourself first (so you know what “broken” looks like), then run the change and confirm it's fixed.

For changes that aren't immediately runnable, the equivalent move is to walk through the agent's output yourself and check it against your goal. “Does this brief actually answer the question I asked?” “Does this email actually say what I want it to say?” The principle: don't trust automated success signals as proof of behavioral success.

Move 5 — Iterate. If anything in moves 1–4 didn't satisfy you, tell the agent what's still wrong and ask for the next pass. Do not start trying to fix it yourself. The agent is the one who wrote the change; the agent is the one who fixes it. Phrase the iteration specifically:

• “This change touches config.py, which I didn't ask about. Please give me a version that only touches tidy.py.”
• “Your change passes the test, but when I run it on a real Downloads folder, files starting with .env end up in the wrong place. Please fix that case and add a test for it.”
• “You removed the line that creates the output folder. I think we still need that. Please put it back, or explain why it's safe to remove.”

The iteration is the back-and-forth that the rest of the course assumes you can do. After a few passes, most changes are clean. After enough real edits, the iteration becomes natural — but the only way to get there is to keep doing it deliberately at the start.

A printable director's review checklist lives at /resources/module-03/diff-review-checklist/. Print it. Use it on every real change you direct in this module. With practice, the five moves compress into a single mental pass — but don't rush that.

Answering the three common objections CORE

When students first hear the five-move review, three objections show up. Worth answering directly because the rule is easier to keep when you understand why it holds.

“This is too slow. I'll never get anything done.” It is slower than auto-accepting, yes. It is not slower than recovering from the bugs auto-accepting introduces. In practice the review adds a small amount of time per change; the bugs it catches can take much longer to find and recover from later. The math is not close.

“The agent is smart enough that I can skip this on small changes.” The catch-rate on small changes is lower than on large ones, because reviewers pay less attention. “Small and obvious” is precisely where silent deletions and plausible-wrongs slip in. The honest rule is the simple one: run the five moves, every time. The exception — “unless it's a throwaway test on a folder I'm about to delete” — is narrow enough to not be worth carving out.

“If I have to review every change anyway, why am I using an agent?” Because reviewing a change is faster than making it. The agent is still doing the work — locating, planning, writing, running tests. You are still getting the time back. The review is the last, cheapest step of a pipeline whose expensive parts the agent absorbed. You're not the typist; you're the director.

The underlying principle is the one the rest of your career rides on: an AI agent can do most of the work much faster than you can, but it cannot take responsibility for the result. The director's job is to take the responsibility the tool cannot. The five-move review is the mechanical form of that responsibility.