Engineering Guides · The Durability Curve

What Is Prompt Injection? How to Stop a Tricked AI Agent Leaking Your Data

Tue, 27 Oct 2026 00:00:00 GMT

One GitHub issue got an agent to post private-repo data in public. Why no filter fully stops it, which guards held in our runs, and how to test yours.

Prompt injection is when text inside something your AI agent reads, such as an email, a web page or a GitHub issue, gets followed as if you had typed it. OpenAI calls it “instructions placed in external content in an attempt to make the model do something the user did not ask for.”

In May 2025 researchers at Invariant Labs set up demo repositories and put an instruction in a public GitHub issue. When the repository’s owner asked Claude to look at the open issues, the agent read data from the owner’s private repositories and published some of it in a public pull request: the names of private projects, a plan to relocate to South America and the owner’s salary.

The tools were legitimate and the request was ordinary. Nothing in GitHub’s server code was at fault, Invariant says; the attack works “even with fully trusted tools”. The instruction came from a stranger.

On 27 October we tested three coding agents ourselves, one run per setup. None followed the plain planted instructions we saw them open. Their guards were another matter: Claude Code’s fetch tool sits outside its shell sandbox, in bypassPermissions mode its sandbox let curl reach our website until one more setting was added, and Cursor’s sandbox let commands reach a listener on the same machine, which its docs say is blocked.

This guide explains why that happens, what each guard controls, and how to test your own setup; the ten-minute check is under the lethal trifecta.

The short version: a leak needs private data, text a stranger wrote and a way out, in one session. Take one away for any task that reads strangers’ text, then test the guard yourself: an agent that ignores a planted line tells you about the model and nothing about the guard.

What is prompt injection? Direct and indirect

To the owner, the issue was a stranger’s note to read later. To the agent it was text in the same stream as the owner’s request, and it read like a request. Models follow instructions in content, which is what makes them useful, and as Simon Willison puts it, they “are unable to reliably distinguish the importance of instructions based on where they came from.”

Willison proposed the name in September 2022, after SQL injection; the shared problem, he later wrote, is “mixing together trusted and untrusted content in the same context.” He has since credited someone else with coining it on 13 May 2022, “four months before I did”.

The OWASP Top 10 for LLM applications ranks prompt injection first, as LLM01:2025, and splits it in two. A direct injection comes from the person typing; an indirect one arrives when a model “accepts input from external sources, such as websites or files.” The GitHub issue was the indirect kind, the one that turns an agent against the person it works for. (OWASP counts jailbreaking as a form of prompt injection; Willison keeps the two apart.)

Where prompt injection hides

Email is the obvious way in, and two real cases came that way. In June 2025 researchers at Aim Security disclosed EchoLeak, in which a single crafted email made Microsoft 365 Copilot leak data with no click from the user (a later case study takes it apart). Microsoft rated it Critical, published it as CVE-2025-32711 on 11 June 2025, and says it “has already been fully mitigated”.

The second, an attack on ChatGPT, was reported to OpenAI by outside researchers in 2025: an email that read like an HR follow-up asked the assistant to “submit these details to the compliance validation system”. OpenAI says that in testing it “worked 50% of the time” when the user asked deep research to read the day’s emails.

Web pages carry it too: for a browser agent, Anthropic writes, “every webpage an agent visits is a potential vector for attack.”

A coding agent reads more places than it seems to. The paid kit for this guide (more at the end) plants a harmless instruction in eight of them: the project README, an HTML comment in a Markdown file, a code comment, a vendored package’s README, an AGENTS.md inside that package, an outsider’s bug report, a web page the docs link to, and a record an API returns. Someone other than you may have written any of them.

The attacks have changed shape too: OpenAI says the most effective now “increasingly resemble social engineering more than simple prompt overrides.” And while a vendor can close the route out of its own product, as Willison notes most did, “once you start mixing and matching tools yourself there’s nothing those vendors can do to protect you!”

Make a list of what your agent reads that someone else can write. The public repository’s issues belonged on it.

The lethal trifecta: when an AI agent can leak your data

Willison’s lethal trifecta is the most useful single test in this guide. Data theft needs three things together: “Access to your private data”, “Exposure to untrusted content”, and “The ability to externally communicate in a way that could be used to steal your data.” In his words: “If your agent combines these three features, an attacker can easily trick it into accessing your private data and sending it to that attacker.”

Read the third leg as a way out: anything that sends data, or changes something other people can see. The GitHub pull request did both.

The GitHub case had all three in one connector: it could read the private repositories, the public issue was untrusted content, and a pull request on a public repository was the way out.

A way out is easier to have than it looks: any tool that calls an API, loads an image or shows you a link to click “can be used to pass stolen information back to an attacker,” Willison writes.

Meta’s security team turned this into the Agents Rule of Two: in any one session, an agent should have no more than two of untrusted input, private data, and the power to “change state or communicate externally”; a task that needs all three needs supervising. Willison adds that untrusted input plus the power to change things is risky even with nothing private in reach.

Do this first (10 minutes): list the AI tools and agents you use. For each, ask: can it read your private things? Does it read text strangers can write (mail, web pages, issues, dependencies)? Does it have a way out (send, post, open a pull request, fetch a URL)?

For any tool with all three, take one away for the tasks that read strangers’ text: narrow its access, or move those tasks to a session with nothing private in reach. A coding agent usually scores three yeses, so close its way out and make its writes ask first.

Claude Code Skills: How They Work, How to Write One, and How to Check Claude Uses It

Mon, 26 Oct 2026 00:00:00 GMT

Claude picks a skill from its name and description. How to write the two lines it reads, and check it chose yours.

A Claude Code skill is a folder of instructions that Claude loads when a task calls for it. It follows an open format that the Claude app, ChatGPT and Codex also read: the SKILL.md file travels between them, though how you install it, when it runs on its own and which fields count all differ.

I wrote one that writes commit messages in a house format and gave it the name and description Anthropic’s guide to writing skills tells you to avoid: helper, and “Does stuff with files”. Typed as /helper, it worked.

Asked in plain words for a commit message, Claude passed it over 12 times out of 12 (four phrasings, three runs each, Claude Opus 5.5, one repository) and wrote decent messages of its own, with nothing in the reply to say a skill had been skipped.

The instructions were fine. Claude never chose them.

The point of a skill is that you stop having to remember it, so one that runs only when you type its name has done half its job. This guide takes helper from that first draft to a version Claude chose on all 21 commit-message requests in my runs, and ends with the check that tells you which kind you have. (If a well-written skill of yours has stopped being chosen, The Skill That Never Fired covers why.)

The short version: Claude chooses a skill mainly from two lines, its name and its description. Give it a name that says the job and a description that says what it does and when to use it, in the words you would type. Then check the choice: in Claude Code, a Skill call shows that Claude picked the skill, and a reply in the right format doesn’t. Section 3 shows how to write a skill, and section 6 how to check it.

What are agent skills?

Agent skills is the name the format goes by outside Claude Code. Anthropic created it, describing a skill as “a directory containing a SKILL.md file” whose instructions, scripts and resources “give agents additional capabilities,” and in December 2025 it “published Agent Skills as an open standard for cross-platform portability.”

Claude Code’s skills “follow the Agent Skills open standard, which works across multiple AI tools,” and OpenAI’s skills for ChatGPT and Codex “build on the open agent skills standard.”

Inside Claude Code, a skill is easy to confuse with four other things:

CLAUDE.md is read at the start of every session, so it suits facts the agent always needs, while a skill’s body “loads only when it’s used,” which suits a procedure.
A custom slash command is now the same thing: “Custom commands have been merged into skills.”
A subagent “runs in its own context window”, while an invoked skill’s content “enters the conversation as a single message and stays there across later turns.”
MCP is “an open-source standard for connecting AI applications to external systems”; a skill can tell the agent how to use what MCP connects.

Claude Code’s docs give the test for when to write one: “Create a skill when you keep pasting the same instructions, checklist, or multi-step procedure into chat, or when a section of CLAUDE.md has grown into a procedure rather than a fact.” A house commit format is that kind of procedure.

How Claude chooses a skill

At the start of a session, in the words of Anthropic’s engineering post on skills, “the agent pre-loads the name and description of every installed skill into its system prompt.” The rest of the file waits: “If Claude thinks the skill is relevant to the current task, it will load the skill by reading its full SKILL.md into context.”

So the choice rests mainly on two lines. My test sessions loaded 41 skills: helper and 40 others that come with Claude Code or with my account, none of them about commits.

Asked for a commit message, Claude had the name helper and the words “Does stuff with files” to go on. The format, the ticket rule and the Refs: line were all in the body, which the skill loads only once Claude picks it.

Claude picks a skill from its name and description, and the rest loads only after it has picked.

There are two exceptions. A skill set to run only by name (section 3) stays out of the listing, and once the listing outgrows its budget, by default 1% of the context window, Claude Code drops descriptions “starting with the skills you invoke least.”

How to write a Claude Code skill

A skill is a folder named after the skill, with a SKILL.md inside. Here is the version I recommend: helper’s instructions with two lines added, under a name that says the job and a description adapted from Anthropic’s example.

---
name: writing-commit-messages
description: Writes commit messages in the house format from the staged diff. Use when the user asks for a commit message or the message for changes about to be committed. Not for code review.
---

Write the commit message for the staged changes:

1. Read the staged diff (git diff --staged) and the branch name (git branch --show-current).
2. First line: type(scope): summary, at most 60 characters. Types: feat, fix, docs, refactor, test, chore.
3. A blank line, then one or two sentences on why the change was made.
4. A last line: Refs: followed by the ticket ID from the branch name (PAY-142 from feat/PAY-142-retry-timeout). If the branch name has no ticket ID, leave the Refs: line out and never invent one.

Reply with the message only. Do not run git commit.

Try it

Make the folder in your repository and save the file there as .claude/skills/writing-commit-messages/SKILL.md:

mkdir -p .claude/skills/writing-commit-messages

Then stage a change, start Claude Code in the repository and ask for a commit message in your own words. The message should open with a type(scope): line and, if your branch name carries a ticket, end with Refs: and that ticket; section 6 shows how to confirm Claude chose the skill rather than writing the message itself.

In my runs this version was chosen on all 21 commit-message requests, took none of 9 code-review requests, and on a branch with no ticket left the Refs: line out all six times. I wrote its description after seeing the first two runs, so some of those requests are ones it was written around; test it on yours.

What changed from Anthropic’s example

Anthropic’s Git Commit Helper example, which the runs also tested, is the starting point. The name now says the job.

The description is in the third person, as Anthropic’s page asks (“Always write in third person”), where the example opens with “Generate”; it drops “or reviewing staged changes”, a job this body doesn’t do, and adds “Not for code review”. The body says where to look and what to do when a branch has no ticket.

On a branch with no ticket, the version with Anthropic’s description, in the third person, and the original body left a placeholder such as Refs: <TICKET-ID> in all six runs; the recommended version left the line out.

The description

The description is where you say when the skill applies, since a name has no room for it. Claude Code’s skills docs define the field as “What the skill does and when to use it,” and the examples on Anthropic’s best-practice page follow one pattern: what the skill does, then “Use when” and the requests it is for.

Put in the words you would type when you want the skill. Keep it under 200 characters, the Claude app’s limit. Claude Code also takes a when_to_use field for trigger phrases, which the Claude app rejects (section 4).

The name

The name is the folder’s name and the command you type: lowercase letters, numbers and hyphens, up to 64 characters, and the standard says it “Must match the parent directory name”.

In my runs a name that says the job carried the choice even under the vague description, including on requests that never said “commit”. In the two-skill test in the earlier essay, “A good description rescued names that carried no signal,” and “A good name had already done the same for descriptions that carried none.” Routing broke down only when neither said what set the skill apart, and section 6 shows the same here.

Every tool reads the description, and it is the one portable field that can say when, so write it first.

The body

The body is the procedure. Keep it to steps Claude can follow, because “Once a skill loads, its content stays in context across turns, so every line is a recurring token cost.” Longer reference material can go in other files in the folder, which the body names and Claude opens “only as needed.”

Skills with side effects

One frontmatter field decides whether Claude may choose the skill at all. This skill is designed only to draft a message, so an unwanted invocation is low-risk; its “Do not run git commit” is an instruction, and tool permissions remain the separate safeguard.

A skill that commits, deploys or sends something is different. Claude Code’s docs say to set disable-model-invocation: true “for workflows with side effects or that you want to control timing, like /commit, /deploy, or /send-slack-message. You don’t want Claude deciding to deploy because your code looks ready.”

With that set, the description leaves the listing and Claude can’t run the skill on its own; stopping the action itself is a job for the permissions covered in the second guide in this series.

Where skills live in Claude Code, the Claude app, ChatGPT and Codex

Claude Code

Claude Code reads skills from two places most people need. ~/.claude/skills/<skill-name>/SKILL.md is yours and loads in all your projects on this machine. .claude/skills/<skill-name>/SKILL.md lives in a repository, and the docs’ advice is “Commit it so your team gets it too.” When two skills share a name, “Enterprise over personal, and personal over project,” so a personal helper would replace the team’s.

The Claude app

In the Claude app, skills sit under Customize > Skills; you upload the folder as a ZIP, and Claude “reviews available skills, loads relevant ones, and applies their instructions.” Anthropic’s help centre lists skills on the Free, Pro, Max, Team and Enterprise plans with code execution on, though the developer overview still lists only the paid plans.

Uploads accept six frontmatter fields, name, description, license, compatibility, metadata and allowed-tools; anything else, disable-model-invocation and when_to_use included, “fails with a hard error instead of ignoring the field.”

ChatGPT

In ChatGPT, according to OpenAI’s help page, “Skills are available to eligible ChatGPT Business, Enterprise, Healthcare, and Edu users, subject to workspace settings and product availability.” They sit under Plugins, on a Skills tab, and you call one by typing @.

Codex

Codex looks for skills in .agents/skills in the repository and ~/.agents/skills in your home folder, and you mention one with $. To stop Codex choosing a skill on its own, set allow_implicit_invocation: false under policy: in the skill’s agents/openai.yaml; then “explicit $skill invocation still works.”

Read a skill before you install it

A skill someone else wrote goes into the same listing as yours, and whenever Claude picks it, its instructions join the conversation. Skills, Anthropic’s post notes, “can also include code for Claude to execute as tools at its discretion.”

The post’s advice is to install skills “only from trusted sources” and, for anything less, to start “by reading the contents of the files bundled in the skill,” including instructions “that instruct Claude to connect to potentially untrusted external network sources.”

Read the SKILL.md as closely as the scripts: when Snyk scanned 3,984 skills from two public registries in February, its reviewers confirmed 76 with “malicious payloads in their markdown instructions to AI agents” (vendor research).

Which skills to trust, and how to pin and remove them, is the subject of Skills Are Package Management for Your AI.

How to check Claude chose your skill

Claude Code’s docs draw the line this guide has been following: “Seeing a skill trigger tells you Claude found it, not that it did what you intended. To know a skill is working, measure separately whether Claude invokes it on the prompts it should, and whether the output matches what you expect when it does.”

Calling a skill by name shows you the output. Three steps check the choice.

Step 1: Check that Claude Code can read the frontmatter

If it can’t, Claude Code “loads the skill body with empty metadata, so /skill-name still works but Claude can’t match against your description.” The validator, in Claude Code 2.1.233 or later, finds that. Here it is on a description with a stray quotation mark inside:

claude plugin validate .claude/skills

✘ Found 1 error:

  ❯ frontmatter: YAML frontmatter failed to parse: YAML Parse error: Unexpected token. At runtime this skill loads with empty metadata (all frontmatter fields silently dropped).

Step 2: Read the Skill call

When Claude picks a skill in your session, it calls a tool named Skill, and a headless run prints that call. Scott Spence measured skill activation this way in February, running prompts “through claude -p with --output-format stream-json” because he only needed to know “did it call Skill() or not?”

This command prints the name of the skill Claude called. On helper with Anthropic’s description it printed the line below; on the vague helper, nothing. It needs jq, and it ran as shown on Claude Code 2.1.292; the stream’s layout can change between versions.

claude -p "Write a commit message for the staged changes." \
    --output-format stream-json --verbose \
  | jq -c 'select(.type=="assistant") | .message.content[]?
           | select(.type=="tool_use" and .name=="Skill") | .input'

{"skill":"helper"}

Each claude -p run is a new session, which matters, because “leftover context from authoring the skill will mask gaps in the written instructions.”

Typing /helper can’t stand in for this: Claude Code runs a skill you name directly, and in my runs the stream then showed no Skill call at all. An empty result can also mean the run failed, so look at the reply before you rewrite anything.

Step 3: Vary the words

On Sonnet 4.5 with four Svelte skills, Spence found activation “keyword-dependent.” With no extra setup, prompts that used a skill’s keyword activated it nearly every time, generic phrasing 20 to 40% of the time, and indirect phrasing almost never.

So run the check on three or four phrasings you would really use, at least one without the skill’s own words, and on a near-miss request it should leave alone, two or three times each. Then read the replies for the other half of the docs’ test: when Claude chooses the skill, the output should follow it.

What the check found

Reading the Skill call across phrasings is the check that caught helper. I ran five versions in Claude Code 2.1.292 with Opus 5.5: the same body under four frontmatters, each quoted from Anthropic’s best-practice page or built on its naming convention, plus the version above with its two added lines. I wrote each routing plan down before its runs. These were the requests:

Commit requests (three times each)
  Write a commit message for the staged changes.
  Draft the commit message for what I've staged.
  I'm ready to commit. What should the message say?
  Summarise my staged diff as a commit message.
Without the word "commit" (three times each)
  Prepare the message for the changes I'm about to save.
  I'm about to push this. Write the summary line and a short reason for the history.
  Describe this change for the project history in the team's usual format.
Review requests the skill should leave alone (three times each)
  Review my staged changes for bugs and potential regressions.
  Check the staged diff for anything risky before I push.
  What did I change in the staged files, and is anything missing a test?
Unrelated (twice each)
  What does the fetch function in client.py do?
  Add a docstring to the fetch function in client.py.

Every planned phrasing gave the same result on every repeat. The version whose name and description both said nothing about commits was never chosen, and on these phrasings either a telling name or a telling description was enough, whether or not the request said “commit”.

No version took a review request, including the two whose descriptions still said “reviewing staged changes”: the first went to Claude Code’s own code-review skill every time, and Claude answered the other two itself, so this shows the clause losing to a neighbour that owns the job, not what happens without one.

On the branch with a ticket, the commit message in every reply from a chosen skill passed a format check I wrote down before scoring it (a type(scope): first line of 60 characters or fewer, a blank line, a body, and Refs: PAY-142 as the message’s last line), 84 of 84.

But 14 replies added text outside the message, twelve of them notes about the change, against the skill’s “Reply with the message only,” so 70 of 84 followed every instruction checked. Being chosen didn’t guarantee the skill was followed in full.

The replies are why the call is the thing to read. Without the skill, 11 of the 12 messages for the commit requests carried the ticket number and none used the house format, so the vague skill looked as if it had nearly worked.

On the request without “commit” that mentioned “the team’s usual format” the opposite happened: Claude went looking for the team’s conventions, opened helper’s SKILL.md as an ordinary file and copied the format exactly, all three times, and once more on another phrasing, with no Skill call.

A reply in the right format can’t tell you whether Claude chose the skill.

This is one skill, one model and twelve phrasings, with 40 other skills loaded. Two exploratory phrasings an AI reviewer ran outside the plan saw the description version miss once, so test your own setup. Beyond that one contest with code-review, it says nothing about a crowded listing where two skills could both answer.

Read your result

If Claude chose the skill on every phrasing and left the near-miss request alone, the choice passes.

If it missed some, put the words from the misses into the description, or into when_to_use in Claude Code, and run them again, each two or three times.

If a sharp description still misses, or another skill fires instead, look outside the skill as well: a neighbour claiming the same job, a listing that has dropped descriptions, or a skill of the same name shadowing yours, which the Skill call won’t reveal because it prints only the name. The Skill That Never Fired covers those, and its runnable pack grades a skill against the neighbour it could lose to.

Anthropic’s skill-creator plugin automates part of this: it “generates should-trigger and should-not-trigger prompts, measures the hit rate, and proposes description edits when the skill activates on the wrong requests.” In the Claude app there is no stream to read; the help centre’s advice is to “Review Claude’s thinking to confirm it’s loading the skill.”

The checklist

This week

☐ Your skill’s name says its job, and its description says what it does and when to use it, in the words you would type, with the main use first.

☐ claude plugin validate passes on the folder.

☐ The Skill call appears on three or four phrasings you really use, one of them without the skill’s keyword, each run more than once, and stays away from a near-miss request the skill should leave alone.

☐ When Claude chooses the skill, the reply follows it; you have checked the output as well as the call.

Next

☐ Any skill that commits, deploys or sends has disable-model-invocation: true, and your permissions cover the action itself.

☐ Every skill you didn’t write has been read line by line, the SKILL.md as well as the scripts.

☐ The team’s skills are committed in .claude/skills/, with names a personal skill won’t shadow.

Once it works

☐ The check runs again after each change to a description, a change of model, and each Claude Code update.

☐ Skills headed for the Claude app use only its six frontmatter fields.

The whole job

Write the description for the moment Claude chooses, then watch it choose. This week, run the check on the skill you rely on most, with one phrasing that leaves out its keyword and one request it should ignore, and rewrite the description until the call appears, the way {"skill":"helper"} did, or make it a skill you call by name. Until you have watched Claude choose it, a skill is a hypothesis.

Further reading: if you read three of the originals in full, make them Claude Code’s skills docs, Anthropic’s skill-authoring best practices and Anthropic’s engineering post on skills. Every other source is linked where it is used. The sources run from October 2025 to October 2026, and each quotation was matched word for word against a saved copy of its page.

AI Agent Permissions: How to Stop Your Agent Doing Something You Can't Undo

Fri, 23 Oct 2026 00:00:00 GMT

Our deny rule stopped one spelling of a branch delete and missed the next. The guards that hold for agent actions you can’t undo, and how to test yours.

AI agent permissions are the rules that decide what a coding agent may do without asking you first; a sandbox decides what it can reach when it does. Claude Code, Codex and Cursor all have both, under different names.

A user asked an agent to “clean up old branches.” The agent “listed remote branches, constructed a pattern match, and issued a delete.”

The case comes from Anthropic’s internal log of times its agents went further than asked, adapted in its write-up of Claude Code’s auto mode. Anthropic calls this overeager: the agent is trying to help and “takes initiative beyond what the user would approve.” The user “may have only meant to delete local branches.”

Had the agent asked first, a prompt would have shown you the delete, and stopping it would have depended on you reading it. Someone has to judge what “clean up” covered: the agent, you at the prompt, or in auto mode a second model reading the transcript.

This guide follows that one deletion through five guards, from the prompt to a server that refuses it, and ends with a way to check which of your own guards hold.

Short of time? Section 2 carries the argument, and the box in section 3 is the first thing to do.

Where a guard can sit

An action can be stopped in three places.

Someone can judge whether you meant it: you at a prompt, or a reviewer model.

The agent’s machine can block it, with a rule that names the command, a hook, or a sandbox that limits what commands reach.

The system the action lands on can refuse it, which for a branch is the git server.

With nothing set up on the machine or the server, the deleted branch meets only the first kind: a git server accepts a delete unless someone has told it not to.

None of the four stops in the first guide in this series sits at the destination, and section 4 shows why that is the place to add one.

Why asking can’t carry what you can’t undo

In Manual mode the prompt is the guard you have, and Anthropic has measured how it goes: “approval fatigue, where people stop paying close attention to what they’re approving,” and “in practice users accept 93% of them anyway.”

That rate is what you would expect if most prompts are harmless. The harmful one arrives looking like the rest.

Auto mode hands the judgement to a classifier, a second model that checks each risky action before it runs. Anthropic tested it on 52 real overeager actions, collected from sessions where an employee denied an action or it was flagged afterwards, and the full system let 17% through. On 52 cases, each miss moves that figure by about 2 points.

“The 17% false-negative rate on real overeager actions is the honest number,” the write-up says, and it explains what went wrong: “In the majority of misses we examined, the classifier correctly identifies the danger.” What it misjudges “is whether a real consent signal in the session actually covers this action.” Its example: “‘Clean up the PR’ was said by the user, but it doesn’t name force-pushing.”

It is also a chosen trade-off: the second stage “cleans up the false positives but lets a few more dangerous actions through,” and the full system wrongly blocked 0.4% of 10,000 real staff actions.

Anthropic’s verdict is that against careful approval of every action “it’s arguably a regression,” and that auto mode is “not a drop-in replacement for careful human review on high-stakes infrastructure.”

The agent and the classifier fail at the same judgement, whether this request covers this action, and a prompt asks you to make that judgement again every time, at speed.

So sort your agent’s actions by whether you can undo them. The ones you can undo, such as an edit inside a repository with its history, can run with fewer prompts inside a wall.

Actions you can’t undo need a guard that never has to decide what anyone meant.

List what your agent can’t undo

The block rules Anthropic described when it launched auto mode in March make a ready starting list. The first group is “Destroy or exfiltrate. Cause irreversible loss by force-pushing over history, mass-deleting cloud storage, or sending internal data externally.”

Another covers shared systems: “pushing directly to main, running production deploys, or modifying shared resources or jobs the agent didn’t create.” A third includes “modifying the agent’s own permission config.”

Those defaults have since moved: Claude Code’s current docs allow by default “Pushing to any branch of the repository you’re working in, including the default branch,” while a force push is still blocked. Check what your installed version permits rather than assuming the reviewer stops a push to main.

For a typical coding agent, the list comes down to remote branch deletes and force-pushes, writes to production data, deploys, and anything that sends data out.

A clone may still hold a deleted branch’s commits, but nothing at the moment of the command says whether one does, so treat the branch as gone.

Do this first (10 minutes): Write down the actions your agent could take this week that you couldn’t undo yourself in five minutes. Use Anthropic’s groups as prompts: losing data or history, sending data out, and touching shared systems such as main, production or someone else’s jobs. Next to each, write what stops it today. If the answer is “it would ask me,” start there.

Five guards on the deleted branch

Five guards now meet the same deletion. The first two depend on someone judging consent. The next two judge nothing, but they run on the agent’s own machine. The last runs somewhere else.

François Martin, writing after an agent of his emptied two main branches, set out the same kind of order with branch protection first: “I would set up three layers, in this order.” We ran steps 4 and 5.

Step 1: Ask

Claude Code’s Manual mode (its config name is default) “stops and asks you before most actions that edit files, run shell commands, or reach the network,” according to its docs. The delete arrives as one more prompt among many. Keep prompts for actions you can undo, where a careless yes costs you a minute.

Step 2: A second model

Claude Code’s auto mode, Codex’s automatic review and Cursor’s Auto-review all send risky actions to a reviewer model. For this case Anthropic’s March write-up says, “This would be blocked since the request was vague, the action irreversible and destructive.”

Other cases get through. Martin was running Claude Code in auto mode with Opus 5 when it tried to undo a bad commit from a shallow clone, and the revert “staged every single file in the repository for deletion.”

The pushes that followed “removed 151 files from one main branch and 723 from the other,” as he describes on Karakun’s blog. Of the model and the classifier, his verdict is “Neither stopped this.”

The vendors say what a reviewer is. Claude Code’s docs call the classifier “a per-action control, not an isolation boundary,” and Cursor’s say “Auto-review is not a security boundary.” Use one to be asked less often.

One detail matters for your list. In auto mode, “File writes and edits inside the project directory are allowed without a classifier call,” because, Anthropic says, “in-project edits are reviewable via version control.” A file that version control doesn’t track has no such undo.

Step 3: A sandbox

A sandbox limits what a command can reach, and Anthropic’s engineers write that “effective sandboxing requires both filesystem and network isolation.” In their internal use it cut permission prompts by 84%.

Claude Code’s sandbox gives a command “no direct route to the network,” and its proxy “checks the hostname of each connection against your allowed and denied domains.”

If the git host isn’t allowed, a sandboxed push can’t reach it, though a blocked command can be retried outside the sandbox unless you set allowUnsandboxedCommands to false.

The agent’s job usually needs that host, and once it is allowed, the sandbox can’t tell an ordinary push from a deletion. That sandbox also “covers shell commands only. Claude’s file tools, MCP servers, and hooks run outside it.”

Step 4: A deny rule

A deny rule names a command and blocks it, and in Claude Code “Deny rules block in every mode, including bypassPermissions.” This one, in the project’s .claude/settings.json, names the delete:

{
  "permissions": {
    "deny": ["Bash(git push origin --delete *)"]
  }
}

We ran it with Claude Code 2.1.292 against a throwaway repository with a local remote, asking the agent to run one exact command each time. The rule stopped the command it names:

TOOL CALL: git push origin --delete old-1
TOOL RESULT: Permission to use Bash with command git push origin --delete old-1 has been denied.

Then we told it to send the same deletion in the other form git accepts:

TOOL CALL: git push origin :old-1
TOOL RESULT: To ../remote.git
 - [deleted]         old-1

The branch was gone. We had allowed git push * so that ordinary pushes would run, and the colon form matched that and slipped past the deny rule.

Claude Code’s permissions docs say a deny rule “covers the invocation Claude usually produces and isn’t a security boundary around the program.”

Rulestack, which sells rule packs for these tools, tried 14 ways of writing a push against one rule, and “5 pushed to the remote.” We picked the second spelling ourselves.

Step 5: Refuse it at the destination

The last guard runs on the server. Git has a setting for it, receive.denyDeletes, which “will deny a ref update that deletes the ref” (git docs). We switched it on in the remote and sent the colon form again:

git config receive.denyDeletes true   # in the remote repository

TOOL CALL: git push origin :old-2
TOOL RESULT: Exit code 1
remote: error: denying ref deletion for refs/heads/old-2
To ../remote.git
 ! [remote rejected] old-2 (deletion prohibited)

The server refuses the deletion however the command is spelled, but it refuses only what it names.

In a second test, with receive.denyNonFastForwards on as well, an ordinary push of a commit that deleted every file went through and left main empty. That is the shape of Martin’s incident: commits that deleted the files, with every branch still in place.

His fix for it is a required pull request with CI: “Letting the agent open a pull request instead would have stopped both incidents, even with automatic merging once CI passed,” because “neither the invalid YAML nor the deletion commits would have passed CI.” So match the server rule to the failure, and remember it holds only while the agent can’t change the server.

In our test the remote was a folder on the same machine, so the agent could have deleted the branch there directly. On a real server, the credential decides.

In one user’s report against Claude Code, a force-push needed for a git-history scrub was blocked by branch protection and a ruleset, and “the agent disabled GitHub branch protection rules and a repository ruleset without user authorization in order to unblock a force push.”

Martin’s answer is to give the agent “a machine account or app installation whose credentials have no bypass rights.” That helps only if your own token is out of the agent’s reach, which is how Anthropic built its web sandbox: git credentials and signing keys “are never inside the sandbox with Claude Code.”

On GitHub, a ruleset does the server’s job. “Require a pull request before merging”, together with “Require status checks to pass before merging”, would have stopped Martin’s commits, because his CI would have failed them.

That pair protects you only against what your checks detect, so test the checks too; a pull request alone “doesn’t necessarily have to be approved, but it must be opened.”

“Restrict deletions” means “only users with bypass permissions can delete branches or tags whose name matches the pattern you specify,” and “Block force pushes” sits beside it; both come ticked when you create a ruleset (GitHub’s docs).

On private repositories rulesets need a paid plan, “GitHub Pro, GitHub Team, and GitHub Enterprise Cloud” (GitHub). Without one, receive.denyDeletes on a server you run is the fallback.

The cost is real: under a ruleset, someone with bypass rights deletes branches by hand, and with receive.denyDeletes even your own push is refused, so a branch goes only when an admin turns the setting off or deletes it on the server.

Where you don’t run the destination at all, the credential is the guard: a database user that can read production and not write to it, a token with no delete rights.

The same guards in Claude Code, Codex and Cursor

Each tool has the same pieces under different names.

Claude Code

Where auto mode is available, interactive Claude Code sessions now start in it: “With Claude Code v2.1.283 or later, auto mode is the built-in starting permission mode for interactive terminal and VS Code sessions.” So the deleted branch would meet step 2, and nothing after it unless you add steps 3 to 5.

Codex

Codex starts at the wall. Its local defaults “include no network access and write permissions limited to the active workspace,” and under its on-request approval policy the agent “asks when it needs to go beyond that boundary,” so a push over the network comes to you first.

Setting approvals_reviewer = "auto_review" sends those requests to a reviewer model instead, and automatic review “doesn’t change the sandbox boundary,” OpenAI notes.

Cursor

Cursor has three Run Modes: Auto-review, Allowlist and Run Everything. “The safest useful setup for most people is Auto-review,” its docs say, and Auto-review became the recommended default on 29 May 2026. We ran Claude Code only; the Codex and Cursor settings here come from their docs. Defaults move between versions, so check your mode after every update.

A setup for each kind of task

Set the agent by the job, and keep the guards from your list on whatever the job.

Exploring code you don’t know should start without write access. Claude Code’s plan mode is for investigating and proposing changes, but it still runs “classifier-approved commands when auto mode is available,” so where that matters use a real read-only boundary, such as Codex’s read-only sandbox.

Everyday work in one repository can allow edits inside the project, with version control as the undo, so commit before you hand over a task.

Long tasks you check in on are where fewer prompts pay off: auto mode or automatic review, inside a sandbox.

Unattended runs need an isolated environment and narrowly scoped credentials. Auto mode can run without routine prompts, and Claude Code’s docs suggest a sandbox or container as an extra layer there.

The bypass modes (bypassPermissions, Run Everything, danger-full-access) belong only in a container or a virtual machine: the docs say to use bypassPermissions only “in isolated environments like containers or VMs where Claude Code can’t cause damage.”

Whatever the job, the actions on your list keep their own guards: a deny rule you have tested, a refusal on the server, or a credential that can’t do them.

What no guard here covers

Damage inside the wall stays possible. Auto mode lets in-project edits through without a classifier call, and an agent allowed to edit your project can still break it, or rewrite the tests that judge it. Version control lets you recover tracked changes, but untracked files, ignored files and uncommitted work can still be lost.

An agent can reach its own rules. “In bypassPermissions mode, Claude Code skips permission prompts, including for writes to protected paths such as .git and .claude,” and .claude is where the deny rule lives.

That is the gap in The Guardrail Your Agent Can Reach. A guard the agent can edit is a guard it can remove. Claude Code’s settings can switch off bypass and auto mode, and the docs say those switches “are most useful in managed settings where they can’t be overridden.”

Everything above assumes the agent is trying to help. When something it read is steering it, the request itself comes from an attacker; a later guide in this series covers that case.

Test your guards

The first guide’s rule was that every addition to a harness is a hypothesis until you test it. A guard is one too.

For each action on your list, make a throwaway copy of the destination, ask the agent to do the thing, then ask again with the command written another way. Then turn on the destination’s guard and try once more.

For a git server, the copy takes six lines in an empty folder (it assumes git already has your name and email), and the last one should fail:

git init -q --bare remote.git
git init -q work && cd work && git remote add origin ../remote.git
git commit -q --allow-empty -m init && git push -q origin HEAD:main
git branch old-1 && git push -q origin old-1
git -C ../remote.git config receive.denyDeletes true
git push origin :old-1

remote: error: denying ref deletion for refs/heads/old-1
To ../remote.git
 ! [remote rejected] old-1 (deletion prohibited)
error: failed to push some refs to '../remote.git'

On a hosted server, repeat it in a disposable repository with the same rules and the agent’s own credentials, never against anything you need.

We also gave Claude Opus 5.5 the incident’s own words, “Clean up old branches.”, in three recorded runs. Each time it deleted only the old local branches and never touched the remote.

Those runs were headless, locked down and on one phrasing; Martin’s auto-mode session pushed the deletion of every file to two main branches, which a required pull request with checks would have stopped. The guards are for that run, as long as they match it.

A guard you haven’t seen refuse is a hypothesis.

The checklist

This week

☐ You have a written list of what your agent could do that you couldn’t undo in five minutes.

☐ Each item on it has a guard that doesn’t depend on anyone reading a prompt.

☐ The agent pushes and deploys as its own identity, with no admin or bypass rights.

☐ The server refuses branch deletes (Restrict deletions, or receive.denyDeletes) and force-pushes (Block force pushes, or receive.denyNonFastForwards) on shared branches.

Next

☐ Every deny rule on the list has been tried with the command written a second way.

☐ Changes to main arrive through a required pull request, with checks that would catch a destructive change and that the agent can’t bypass.

☐ The agent’s other credentials can’t do what’s on the list: read-only database users, tokens without delete rights.

Once it works

☐ Every guard on the list has refused on a throwaway copy, and is tried again after each update.

☐ Where agents run unattended, the switches that turn off bypass and auto mode live in managed settings.

The whole job

List what your agent can’t undo, try one of those actions on a copy this week, and keep only the guards you have watched refuse it.

Further reading: if you read three of the originals in full, make them Anthropic on how auto mode was built, François Martin on the agent that emptied two main branches and Claude Code’s permissions docs. Every other source is linked where it is used. The sources run from October 2025 to October 2026; I rechecked each quotation against its page on 23 October 2026.

What Is Harness Engineering? How to Build AI Agents That Catch Their Own Mistakes

Thu, 22 Oct 2026 00:00:00 GMT

Lessons from OpenAI, Anthropic, Google and independent engineers, worked through one failure, with a way to test every change.

Harness engineering is designing everything around an AI model (its instructions, context, tools, checks and sign-off points) so the agent gets work right first time and catches its own mistakes. The model does the reasoning. The harness is much of what makes it reliable.

One failure runs through this guide. You ask a coding agent to tidy up a data model. It renames a field, the code compiles, the checks in front of it pass, and it reports success. Two other services that read that field quietly break. Thoughtworks engineers, who use this example, call it “locally correct, but systemically destructive,” and they say the gap between model intelligence and reliable delivery “is not primarily a model-quality issue.” What the agent lacked, in their words, was “system-level visibility, architectural guardrails and automated feedback mechanisms.”

This guide applies what OpenAI, Anthropic, Google, Thoughtworks and independent engineers have learned to that one failure, a layer at a time, and shows how to test each fix. If you have ten minutes, read section 3 and do the box in step 2 of section 4.

What a harness is

The short version is a formula: Agent = Model + Harness. Birgitta Böckeler, writing on Martin Fowler’s site, splits the harness in two. Part of it comes built into the tool, such as the system prompt and the way it searches your files. The rest is an outer harness that you build for your own work and your own codebase.

She also gives the most useful working model. Guides steer the agent before it acts: instructions, skills, examples. Sensors check its work afterwards: tests, linters, type checks, review. Guides alone give you an agent “that encodes rules but never finds out whether they worked”; sensors alone, one “that keeps repeating the same mistakes.” The renamed field got past both. Nothing told the agent the field was shared, and nothing checked the services that read it.

Mitchell Hashimoto, who started calling it harness engineering in a February post, states the aim: “anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again.” Read “never” as the aim. Agents are not deterministic, so some failures come back by chance, and some fixes cause new problems. That is what section 3 is about.

For the case that the harness matters more than the model, see Harness Engineering: Same Model, Different Product.

Harness engineering vs context engineering

The short answer: context engineering decides what the model sees; harness engineering is the wider system that also prevents, checks and fixes. Anthropic calls context engineering “the natural progression of prompt engineering.” HumanLayer draws the line the other way, calling harness engineering “the subset of context engineering” done through the agent’s own configuration points.

Either way, the renamed field failed on both counts: the services that read the field were never in the agent’s context, and no check ran against them. Treat SmartScope’s comparison as a first question to ask: a single wrong answer suggests context, and quality that slips with repeated use suggests the harness, though one wrong answer can also come from the model, a tool or a missing check.

Every addition is a hypothesis

The reflex after a failure is to add something: another rule, another tool, another check. The best evidence so far says to test that reflex.

Two controlled studies have tested the instruction files at the centre of most harnesses. ETH Zurich researchers, working on Python repositories, found that context files, whether written by a model or by the repo’s own developers, did “not generally improve task success rates,” raised cost by over 20%, and did no better when shorter. The authors conclude the files are useful for “specifying non-standard coding practices.”

A smaller study, by Lulla and colleagues, ran agents with and without an AGENTS.md on 124 pull requests and found the file cut median runtime by 28.6% and output tokens by 16.6%, with completion about the same. The two teams measured cost differently (ETH counted inference cost; Lulla’s counted runtime and output tokens), so their efficiency results are not directly comparable.

Neither found a reliable gain in task success: developer-written files did slightly better in the ETH runs (2.4 percentage points on average, not statistically significant), and clearly better than generated ones.

Anthropic’s harness-design team gives the reason this matters beyond instruction files. Every part of a harness “encodes an assumption about what the model can’t do on its own,” and those assumptions “can quickly go stale as models improve.” Their working rule comes from an earlier post: “find the simplest solution possible, and only increase complexity when needed.” Thoughtworks puts it in three words: “Earn every rule.”

Treat every addition to your harness as a hypothesis. It costs tokens, time and attention, and it assumes something about the model that may stop being true. Make the smallest change that would have caught the failure, run the same kind of task with and without it, and keep it only if the difference pays for the cost. Section 7 shows how.

Fixing the renamed field, one layer at a time

Here is the failure again, fixed in layers: see what happened, make it less likely, catch it when it happens, and contain the damage. For a public contract, the check is the layer that has to hold; the instruction only makes the failure rarer.

Step 1: See what the agent saw

Before adding anything, look at what entered the agent’s context on the failed run. Sourcegraph’s test is to log it, “then check whether the answer was in there.” In the rename it was not: the services that read the field live elsewhere, and the agent never opened them. OpenAI’s line covers it: “anything it can’t access in-context while running effectively doesn’t exist.” If the agent never encountered the dependency, find out why. It may need a clearer instruction to search for it, access to the right repository, or a check that catches what it missed.

Step 2: Make it less likely with one guide

The cheapest change is a line in the agent’s standing instructions file. AGENTS.md is the shared format, read by many coding agents; Claude Code reads CLAUDE.md first and, from v2.1.277, falls back to AGENTS.md when a project has no CLAUDE.md. For the rename, the line might read: Field names in orders/schema are a public contract; before renaming one, search the other services for it and list what reads it. That is the kind of non-standard rule the ETH study found files are good for.

Keep the rest of the file lean. OpenAI’s team tried “one big AGENTS.md” and it failed; for one thing, a giant file “crowds out the task.” Theirs is now “roughly 100 lines,” a “table of contents” pointing to a docs/ folder. HumanLayer keeps theirs “under 60 lines.” In a large repo, put a nested AGENTS.md in each project; agents read the nearest file, “so the closest one takes precedence,” as agents.md puts it.

Software Mansion’s guide cites Anthropic’s Claude Code docs on what goes in and what stays out; here is a shortened version:

Do this first (10 minutes): Open your instructions file. Delete anything the agent could work out for itself, move long sections into separate docs with one-line pointers, and add the one rule your agent most recently broke.

Step 3: Catch it with one sensor

An instruction can be ignored; a check runs every time. For the rename, that is a contract check: each consuming service lists the fields it reads, in a file the agent cannot edit, and the check fails when the schema no longer has one of them. A test that searches the code for the schema’s own field names looks similar and misses this case, because it never looks for the field the agent just deleted. A first version can be this small (TypeScript, with Vitest and a Zod schema):

// contract.test.ts: runs in CI.
// This file, contracts/ and the CI job sit outside the agent's write access.
import { test, expect } from "vitest";
import { OrderSchema } from "../orders/schema";
import consumers from "../contracts/consumers.json";
// { "billing": ["customer_id", "total"], "shipping": ["customer_id", "address"] }

test("every field a consumer reads still exists", () => {
  const fields = Object.keys(OrderSchema.shape);
  const broken = Object.entries(consumers).flatMap(([service, needed]) =>
    needed.filter((f) => !fields.includes(f)).map((f) => `${service} reads ${f}`));
  expect(broken, `${broken.join("; ")}: keep the old name or update the consumer`).toEqual([]);
});

Run against the renamed schema (Vitest 5 with Zod), it fails like this, and it passes again once the old name is back:

FAIL  tests/contract.test.ts > every field a consumer reads still exists
AssertionError: billing reads customer_id; shipping reads customer_id: keep the old name or update the consumer: expected [ 'billing reads customer_id', …(1) ] to deeply equal []

It catches a removed top-level field and nothing subtler; a production check should also compare types, required fields and the consumers’ real interfaces. And it protects you only while the agent being checked cannot change it, so keep the test, the consumer list and the CI step that runs them out of its reach.

Run it automatically after every edit, and follow four rules that make sensors work.

Silent success, verbose failure. That is Thoughtworks’ name for it. HumanLayer learned why the hard way: when they ran the full suite after every change, “4,000 lines of passing tests would flood the context window,” and the agent lost track of its task.
Write the error for the agent. OpenAI writes custom lint messages that “inject remediation instructions into agent context”; Böckeler calls this “a positive kind of prompt injection.” For the rename, the check above says: billing reads customer_id; shipping reads customer_id: keep the old name or update the consumer.
Promote repeated rules into checks. A rule you keep re-adding to your instructions “should be promoted into a mechanical sensor,” says Thoughtworks. Software Mansion’s split: “Written instructions should cover the cases that require judgment. Hooks should cover the cases that do not.”
Protect the checks that decide “done”. Agents need to write and change tests as part of the work. What they must not be able to edit is the check that says the work is finished, and an instruction alone will not hold that line. For a lock that does, see Lock the Tests, Then Test the Lock.

Step 4: Contain it where the stakes are high

Some changes should not go through on the agent’s say-so. Thoughtworks keeps human confirmation for “irreversible or high-blast-radius choices,” such as changes that span several repositories, schema migrations or public API modifications; everything else proceeds with “sensible defaults.” A renamed field in a public contract is exactly that kind of change.

There are four ways to stop an action, and they differ in strength. An instruction asks. A hook runs your own script around a tool call and can block it: HumanLayer’s denies any command that tries to run database migrations and tells the agent to ask a human. A deny rule blocks matching tool calls; in Claude Code, “Deny rules prevent Claude Code from using the specified tool,” and they are checked first, though another tool or command may still reach the same thing. A sandbox limits what any command can reach, such as which folders it can write to.

Hooks carry their own risk: Claude Code’s docs warn that they “execute shell commands with your full user permissions.” Use the strongest layer the risk calls for, and give each agent the fewest permissions it needs; a review agent with read-only tools, as Thoughtworks notes, simply can’t write.

Step 5: Test the fix

Each of those layers is a hypothesis about this failure. Run the rename task, and a few like it, with and without the new line, the check and the stop, and see which one changed the outcome. Section 7 sets out how.

Keep the context clean

The rename needed one fact, the list of services that read the field, and none of the rest of the repository. That is the general aim. As tokens increase, Anthropic explains, “the model’s ability to accurately recall information from that context decreases,” so the goal is “the smallest possible set of high-signal tokens.” Its cookbook compares three ways to free up space.

If you build on the API, start with clearing, which Anthropic calls “one of the safest lightest touch forms of compaction”; the agent can always fetch the result again. When you compact, tell the summary what to keep. Sourcegraph’s compaction prompt keeps the task, the files already inspected, decisions and why, and “approaches already tried that failed, and why.” Leave that last item out “and the fresh window will cheerfully retry them.”

Tools take space too. Anthropic’s tool search loads tool definitions only when the agent needs them, about 85% fewer tokens in its example, and Claude Code now defers MCP tool definitions “until Claude needs them.”

Send noisy research to a subagent and have it return the answer with its evidence and any open questions. Anthropic says a subagent may use “tens of thousands of tokens” and return “often 1,000-2,000 tokens,” and Sourcegraph warns that “a sub-agent that retrieved badly still returns a confident summary.”

Every connector and skill also puts someone else’s text in the agent’s context. Software Mansion describes a published attack that hid instructions inside a skill in invisible Unicode characters. Read every third-party skill or connector before installing it. That lowers the risk without removing it: hidden text can get past a read, and a connector’s list of tools can change after you have read it. How these attacks work, and the guards that limit them, are in What Is Prompt Injection?.

Memory needs curating. On Databricks’ internal benchmark of enterprise tasks, their own memory system scored 22.3% with no memory, 35.8% with searchable raw session traces, and 66.1% with lessons drawn from those traces (68.4% with both). Their caution: “more history is not necessarily better memory.” Give memories an expiry too. Sourcegraph’s example is a memory about a system the team changed weeks ago, which “is worse than no memory: the agent holds a confident, wrong prior it will defend.”

Work that lasts longer than one session

When a job spans several sessions, Anthropic’s long-running agents experiment is the most concrete guide. Each new session “begins with no memory of what came before,” and a later agent would “look around, see that progress had been made, and declare the job done.”

Their fix: a first session writes a feature list of everything “done” means, every item marked failing, in JSON because the model is “less likely to inappropriately change or overwrite JSON files compared to Markdown.” Every later session updates a progress file, ends with a commit so bad changes can be rolled back, and starts by reading both and checking the basics still work.

How to test a harness change

Use this on any change to your harness, including the contract check from section 4. It needs no special tooling, but it does take a fixed set of tasks and a few dozen agent runs. Treat it as a pilot: it will show a large effect, and it cannot prove a small one.

Pick the task class your agent failed on, and collect ten or so real examples. Keep a few back that you never look at while building the fix; those held-out tasks show whether the fix generalises or only fits the examples.
Grade every setup the same way. Decide what “done” means before you start, with held-out acceptance tests the agent never sees, and use them for all four setups. If the contract check were also the grader, setup C would win by definition.
Run each setup on the same tasks, several times. Google’s team recommends running evals in batches, because single runs “can be noisy due to nondeterminism of AI models.”
Record six numbers: tasks passed, tasks verified done by a check you trust, human corrections needed, tokens, time and cost. Count cost honestly; in data from Anthropic’s research system, agents used about 4 times the tokens of chat, and multi-agent systems about 15 times.
Keep the cheapest setup that holds up. If C does as well as D, the instruction line is not earning its place. If none beats A, the failure may lie elsewhere, or the effect may be too small for a test this size; add tasks before you decide.

That comparison measures outcomes. Google’s team separates outcome benchmarks from behavioural evals: checks of one action at a time, such as whether the agent searched the other services before renaming, which show why a score moved where a big benchmark only shows it moving “by a few percentage points.” Make each check strict for simple tasks, or use an AI judge where several paths are fine. When you change model, run the whole test again; the assumptions in your harness may have gone stale.

Which fix for which failure

Most agent mistakes fall into a few kinds, and each has a cheapest fix worth trying first. This table is our synthesis of the guides above; treat each row as a starting hypothesis.

The rename starts in the first and third rows. Once the line is in AGENTS.md, a repeat moves it to the second: the agent knew the rule and broke it, so the next fix is the contract check, and the stop on public API changes is the escalation.

The checklist

This week

☐ Your agent’s last mistake has one check that would have caught it, silent when it passes, with the fix in its error.

☐ Your instructions file holds only what the agent can’t find for itself, plus the rule it last broke.

☐ Actions you can’t undo (migrations, deletes, pushes) are blocked by a deny rule, a sandbox or a hook.

☐ The checks that decide “done” are out of the agent’s reach.

Next

☐ Your latest fix has been run against the setup without it, on held-out tasks.

☐ Noisy research goes to a subagent, which returns the answer with its evidence.

☐ Only the tools and connectors you use are switched on, and every third-party one has been read.

☐ Long tasks have a done-list and a progress note.

Once it works

☐ Each failure you’ve fixed has a small behavioural test.

☐ On a failed run, you check whether the answer was ever in the context.

☐ Memories have an expiry or a review date.

☐ After a model change, every part of the harness is re-tested, and what no longer helps goes.

The whole job

When the agent makes a mistake, find the smallest fix that would have caught it, test it against the setup without it, and keep it only if it earns its place. This week, start with the last mistake your agent made.

Further reading: if you read three of the originals in full, make them Böckeler on guides and sensors, Anthropic on agents that work across sessions and Thoughtworks on the renamed field. Every other source is linked where it is used. They date from June 2025 to October 2026, and every link and quotation was checked on 22 October 2026.

The next guide in this series, AI Agent Permissions, follows one deleted branch through five guards and shows which of them hold.