---
title: "Claim Ledger: What Is Harness Engineering? How to Build AI Agents That Catch Their Own Mistakes"
description: "Lessons from OpenAI, Anthropic, Google and independent engineers, worked through one failure, with a way to test every change."
author: "Harry Floyd"
publication: "The Durability Curve"
canonical: "https://durabilitycurve.com/claims/what-is-harness-engineering/"
essay: "https://durabilitycurve.com/blog/what-is-harness-engineering/"
published: "2026-10-22"
ledger_date: "2026-10-22"
claims: 85
removed_before_publication: 0
---

# Claim Ledger: What Is Harness Engineering? How to Build AI Agents That Catch Their Own Mistakes

*Lessons from OpenAI, Anthropic, Google and independent engineers, worked through one failure, with a way to test every change.*

Essay: https://durabilitycurve.com/blog/what-is-harness-engineering/  
Ledger (canonical, cite this): https://durabilitycurve.com/claims/what-is-harness-engineering/  
Published 2026-10-22 · last verified 2026-10-22

## What the essay claims

Much of an agent's reliability comes from what surrounds the model (instructions, context, tools, checks, sign-off points), and the published guides agree on a short set of steps. But every addition to a harness costs something and encodes an assumption, and the only controlled tests of instruction files found no reliable success gain. So a harness is built one tested fix at a time, against the simpler setup it replaces. The reader starts believing either that reliability is a model-choice problem, or that more rules always help.

## The claim ladder

What the ladder does not reach: none is a controlled comparison of harness against model; no step is shown to transfer to every tool or codebase (OpenAI says its results depend on its repo's structure); the token multipliers (4× / 15×) are Anthropic's own data on its research system. The guide reports what the sources did and recommends; it measures nothing itself.

| # | Claim | Whose behaviour it measures | Status / scope |
|---|---|---|---|
| 1 | Harness engineering is designing everything around the model so the agent gets work right first time and fixes its own mistakes | Definition, synthesised from Böckeler, Hashimoto, Thoughtworks | Definitional; no measurement |
| 2 | Shorter, hand-written instruction files work better than long or auto-generated ones | OpenAI's team, HumanLayer, Software Mansion (practitioner reports) | Reported practice, not a controlled test |
| 3 | Context quality falls as context grows; clearing, compaction and notes each trade something away | Anthropic (engineering post + cookbook) | Vendor guidance |
| 4 | Curated memory raised an agent's pass rate 22.3% → 68.4% with 31% fewer tokens | Databricks' own system on Databricks' internal benchmark | Vendor benchmark; one system, one benchmark |
| 5 | Silent passing checks and agent-readable error messages keep agents on task | HumanLayer (4,000-line incident), OpenAI (custom lint messages), Thoughtworks | Practitioner reports |
| 6 | A feature list + progress file + commit per session lets agents finish long tasks | Anthropic's long-running agents experiment (one test app) | One vendor experiment |
| 7 | Behavioural evals on one failure mode at a time, read as a trend | Google's Gemini CLI team | Vendor method |
| 8 | Context files did not significantly change task success (generated: marginal negative; developer-written: marginal gain) and raised cost by over 20% | ETH Zurich (Gloaguen et al.), Python repos, several agents and models | Controlled study; Python only; not significant either way |
| 9 | An AGENTS.md cut median runtime 28.6% and output tokens 16.6% with comparable completion | Lulla et al., 10 repos, 124 PRs | Controlled study, small; efficiency only |
| 10 | Every harness component encodes an assumption about what the model can't do, which can go stale as models improve | Anthropic's harness-design team (own experience) | Vendor guidance |
| 11 | Tool search defers tool definitions until needed (Anthropic: ~85% fewer tokens in their example; Claude Code defers MCP tools) | Anthropic docs and engineering post | Vendor documentation |
| 12 | Hooks run with the user's full permissions; deny rules and sandboxing are separate layers | Claude Code docs | Vendor documentation |

## What would make it wrong

The guide is wrong if a reader who follows the checklist (one-screen instructions file, silent checks with fix-carrying errors, tests the agent cannot edit, a done-list for long tasks) sees no change in how often their agent declares unfinished work done.

## The evidence, row by row

Labels: VERIFIED = checked against the primary source itself by a checker who did not draft the piece, usually with the exact words, where they sit and the date read · EXECUTED = a run or observation done for the piece (code, a query, a count, or a check made live in an app); the claim is what it returned; run records are not published · CHECKED = checked against its source by a separate checker in the older ledger format, which recorded no quote, locator or date · REPORTED = not verified word for word against a primary source (secondary, not openable in full, or supported only in part); the essay words it accordingly · STRUCK = drafted, checked and removed before publication · EXCLUDED = considered and deliberately left out.

### 1. Thoughtworks describes an agent renaming a field that two downstream services consume; calls it locally correct but systemically destructive

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-1
- Quote: "locally correct, but systemically destructive"
- Source: Thoughtworks (Nandyala, Pani), 29 Sep 2026, opening section
- URL: https://www.thoughtworks.com/insights/blog/architecture/engineering-the-harness-a-practical-pattern-for-reliable-coding-agents

### 2. Thoughtworks says the gap is not mainly about model quality

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-2
- Quote: "is not primarily a model-quality issue"
- Source: Thoughtworks, opening section

### 3. Agent = Model + Harness

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-3
- Quote: "Agent = Model + Harness"
- Source: Böckeler, martinfowler.com, 2 Apr 2026 (also Thoughtworks), early section
- URL: https://martinfowler.com/articles/harness-engineering.html

### 4. Part of the harness is built into the tool (system prompt, code retrieval); users build an outer harness for their use case

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-4
- Quote: "outer harness"
- Source: Böckeler, "Metaphors only go so far" area

### 5. A good outer harness raises first-time success and gives a self-correcting feedback loop

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-5
- Quote: "self-corrects as many issues as possible before they even reach human eyes"
- Source: Böckeler, after Figure 1

### 6. Hashimoto's definition of harness engineering; he says in this post he had started calling the practice harness engineering

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-6
- Quote: "is the idea that anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again"
- Source: Hashimoto, My AI Adoption Journey, 5 Feb 2026, Step 5
- URL: https://mitchellh.com/writing/my-ai-adoption-journey

### 7. Anthropic calls context engineering the next step after prompt engineering

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-7
- Quote: "the natural progression of prompt engineering"
- Source: Anthropic, 29 Sep 2025, intro
- URL: https://www.anthropic.com/research/effective-context-engineering-for-ai-agents

### 8. Context engineering targets one output; harness engineering targets quality over time

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-8
- Quote: (paraphrase of comparison table: "A single output is inaccurate" vs "Quality degrades over time")
- Source: SmartScope, Feb 2026, restructured Mar 2026, comparison table
- URL: https://smartscope.blog/en/blog/harness-engineering-overview/

### 9. AGENTS.md is a shared format; nearest file wins in nested repos

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-9
- Quote: "so the closest one takes precedence"
- Source: agents.md, FAQ / monorepo
- URL: https://agents.md/

### 10. Claude Code reads AGENTS.md only when no CLAUDE.md exists (default)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-10
- Quote: "By default, Claude reads AGENTS.md only when you have no CLAUDE.md in your working directory or above it"
- Source: Claude Code docs, When Claude Code reads AGENTS.md
- URL: https://code.claude.com/docs/en/memory

### 11. OpenAI tried one big AGENTS.md and it failed

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-11
- Quote: "one big AGENTS.md"
- Source: OpenAI (Lopopolo), 11 Feb 2026, We made repository knowledge the system of record
- URL: https://openai.com/index/harness-engineering/

### 12. A giant file crowds out the task

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-12
- Quote: "crowds out the task"
- Source: OpenAI, same

### 13. Too much guidance becomes non-guidance

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-13
- Quote: "When everything is 'important,' nothing is."
- Source: OpenAI, same

### 14. OpenAI's AGENTS.md is about 100 lines and acts as a table of contents pointing to docs/

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-14
- Quote: "roughly 100 lines"; "the table of contents"
- Source: OpenAI, same

### 15. Map, not manual

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-15
- Quote: "give Codex a map, not a 1,000-page instruction manual"
- Source: OpenAI, same

### 16. What the agent can't access in context doesn't exist for it

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-16
- Quote: "anything it can't access in-context while running effectively doesn't exist"
- Source: OpenAI, Agent legibility is the goal

### 17. HumanLayer's CLAUDE.md is under 60 lines; craft by hand, don't auto-generate

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-17
- Quote: "under 60 lines"
- Source: HumanLayer (Kyle), 12 Mar 2026, CLAUDE.md section
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 18. /init output brings little value

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-18
- Quote: "tends to produce documents that bring little meaningful value"
- Source: Software Mansion, updated 22 Jun 2026, AGENTS.md
- URL: https://agentic-engineering.swmansion.com/becoming-productive/harness-engineering/

### 19. Put-in / leave-out table adapted from Anthropic's Claude Code docs, as quoted by Software Mansion

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-19
- Quote: (adapted table)
- Source: Software Mansion, AGENTS.md: good and bad practices

### 20. Recall falls as context grows

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-20
- Quote: "the model's ability to accurately recall information from that context decreases"
- Source: Anthropic context engineering, context rot
- URL: https://www.anthropic.com/research/effective-context-engineering-for-ai-agents

### 21. Aim for the smallest high-signal set

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-21
- Quote: "the smallest possible set of high-signal tokens"
- Source: Anthropic context engineering, same

### 22. Three techniques: tool-result clearing, compaction, notes/memory; clearing swaps old results for a placeholder and the agent can re-fetch

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-22
- Quote: (paraphrase)
- Source: Claude Cookbook (Isabella He), 20 Mar 2026, summary table / clearing
- URL: https://platform.claude.com/cookbook/tool-use-context-engineering-context-engineering-tools

### 23. Clearing is a light, safe form of compaction

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-23
- Quote: "one of the safest lightest touch forms of compaction"
- Source: Anthropic context engineering, compaction
- URL: https://www.anthropic.com/research/effective-context-engineering-for-ai-agents

### 24. Sourcegraph's compaction prompt preserves task, files inspected, decisions, failed approaches

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-24
- Quote: "approaches already tried that failed, and why"; "and the fresh window will cheerfully retry them"
- Source: Sourcegraph (Tanner), 15 Sep 2026, compaction prompt
- URL: https://sourcegraph.com/blog/context-engineering

### 25. Databricks' own memory system (MemCurrent) on its internal benchmark of enterprise tasks: 22.3% (no memory) → 68.4% pass rate, 31% fewer tokens, 26% faster than no memory; Mem0 reached 43.9% on the same benchmark

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-25
- Quote: "more history is not necessarily better memory"
- Source: Databricks, 20 Oct 2026, intro + Figure 6
- URL: https://www.databricks.com/blog/memcurrent-designing-persistent-memory-production-agents

### 26. A stale memory is worse than none

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-26
- Quote: "is worse than no memory: the agent holds a confident, wrong prior it will defend"
- Source: Sourcegraph, memory invalidation (JWT example)
- URL: https://sourcegraph.com/blog/context-engineering

### 27. Mechanism rule of thumb (AGENTS.md, skills, MCP, subagents, hooks)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-27
- Quote: (paraphrased table)
- Source: Software Mansion, When to use what
- URL: https://agentic-engineering.swmansion.com/becoming-productive/harness-engineering/

### 28. Role-based subagents don't work; use them for context control

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-28
- Quote: "It doesn't work."
- Source: HumanLayer, Sub-Agents Are for Context Control
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 29. Subagents may use tens of thousands of tokens and return 1,000-2,000

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-29
- Quote: "tens of thousands of tokens"; "often 1,000-2,000 tokens"
- Source: Anthropic context engineering, sub-agent architectures
- URL: https://www.anthropic.com/research/effective-context-engineering-for-ai-agents

### 30. Too many tools is bad

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-30
- Quote: (heading "Too Many Tools Is Bad")
- Source: HumanLayer, MCP Servers section
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 31. Ambiguous tool choice defeats agents too

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-31
- Quote: "If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better."
- Source: Anthropic context engineering, tools
- URL: https://www.anthropic.com/research/effective-context-engineering-for-ai-agents

### 32. Don't connect untrusted MCP servers

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-32
- Quote: "never connect to one you don't trust. This can be a dangerous vector for prompt injection!"
- Source: HumanLayer, MCP warning
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 33. MCP spec: clients must treat tool annotations as untrusted unless from trusted servers

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-33
- Quote: "clients MUST consider tool annotations to be untrusted unless they come from trusted servers"
- Source: MCP specification 2025-06-18, Tool definition
- URL: https://modelcontextprotocol.io/specification/2025-06-18/server/tools

### 34. Published attacks: a skill step fetched and ran ransomware; hidden Unicode instructions in a skill

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-34
- Quote: "visual source review alone may miss malicious behavior"
- Source: Software Mansion, Skills: Security considerations
- URL: https://agentic-engineering.swmansion.com/becoming-productive/harness-engineering/

### 35. HumanLayer auto-denies migration commands and tells the agent to ask the user

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-35
- Quote: (paraphrase: "we automatically deny any Bash() tool calls that try to run migrations, with an instruction to ask the user to run them instead")
- Source: HumanLayer, Hooks

### 36. Guides steer before, sensors check after

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-36
- Quote: "Guides (feedforward controls)"; "Sensors (feedback controls)"
- Source: Böckeler, Feedforward and Feedback
- URL: https://martinfowler.com/articles/harness-engineering.html

### 37. Guides alone / sensors alone failure modes

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-37
- Quote: "that encodes rules but never finds out whether they worked"; "that keeps repeating the same mistakes"
- Source: Böckeler, same

### 38. The rule is called silent success, verbose failure

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-38
- Quote: "silent success, verbose failure"
- Source: Thoughtworks, Sensors
- URL: https://www.thoughtworks.com/insights/blog/architecture/engineering-the-harness-a-practical-pattern-for-reliable-coding-agents

### 39. Full test suite output flooded context; the agent lost track

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-39
- Quote: "4,000 lines of passing tests would flood the context window"
- Source: HumanLayer, Verification
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 40. Custom lint messages carry fixes

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-40
- Quote: "inject remediation instructions into agent context"
- Source: OpenAI, Enforcing architecture and taste
- URL: https://openai.com/index/harness-engineering/

### 41. Böckeler calls this positive prompt injection

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-41
- Quote: "a positive kind of prompt injection"
- Source: Böckeler, sensors
- URL: https://martinfowler.com/articles/harness-engineering.html

### 42. Repeated prose rules should become sensors

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-42
- Quote: "should be promoted into a mechanical sensor"
- Source: Thoughtworks, sensors
- URL: https://www.thoughtworks.com/insights/blog/architecture/engineering-the-harness-a-practical-pattern-for-reliable-coding-agents

### 43. Instructions for judgement, hooks for the rest

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-43
- Quote: "Written instructions should cover the cases that require judgment. Hooks should cover the cases that do not."
- Source: Software Mansion, Hooks
- URL: https://agentic-engineering.swmansion.com/becoming-productive/harness-engineering/

### 44. In Anthropic's long-running setup, later sessions could only flip a feature-list item's passes flag, under an instruction not to remove or edit tests (the 'tests' are feature-list entries)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-44
- Quote: "unacceptable to remove or edit tests"
- Source: Anthropic long-running agents, 26 Nov 2025, Feature list
- URL: https://www.anthropic.com/research/effective-harnesses-for-long-running-agents

### 45. Each new session starts with no memory; engineers-in-shifts analogy

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-45
- Quote: "begins with no memory of what came before"
- Source: Anthropic long-running, The long-running agent problem

### 46. Two failure modes: doing too much at once; later agents declaring the job done

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-46
- Quote: "to do too much at once"; "declare the job done"
- Source: Anthropic long-running, same

### 47. Initializer writes 200+ features, all marked failing; JSON chosen over Markdown

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-47
- Quote: "less likely to inappropriately change or overwrite JSON files compared to Markdown"
- Source: Anthropic long-running, Feature list

### 48. Progress file + git commits allow reverting bad changes; start-up routine reads git log and progress, picks top feature, runs a basic test

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-48
- Quote: (paraphrase)
- Source: Anthropic long-running, Incremental progress; Getting up to speed

### 49. Testing as a human user caught bugs not visible in code

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-49
- Quote: "as a human user would"; "that weren't obvious from the code alone"
- Source: Anthropic long-running, Testing

### 50. Human gates for irreversible or high-blast-radius choices (multi-repo scope, schema migrations, public API changes); otherwise sensible defaults

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-50
- Quote: "irreversible or high-blast-radius choices"; "sensible defaults"
- Source: Thoughtworks, Guides: human gates
- URL: https://www.thoughtworks.com/insights/blog/architecture/engineering-the-harness-a-practical-pattern-for-reliable-coding-agents

### 51. Least-privilege tools: verification agents get read-only tools

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-51
- Quote: (paraphrase: "provisioned with read-only tools, literally lacking file-writing capabilities")
- Source: Thoughtworks, Least-privilege tools

### 52. Before You Trust Your AI Agent kit is free, seven checks, one page each

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-52
- Quote: "Seven checks, one page each"
- Source: The Durability Curve, 9 Oct 2026, subtitle
- URL: https://harryfloyd.substack.com/p/before-you-trust-your-ai-agent

### 53. Scores move a few points with no idea why

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-53
- Quote: "by a few percentage points, and have no idea why it changed"
- Source: Google Developers Blog (Mullen, Gunderman), 9 Sep 2026, intro
- URL: https://developers.googleblog.com/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/

### 54. Behavioural evals check discrete actions (clarifying question, validator before done)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-54
- Quote: (paraphrase)
- Source: Google, behavioural eval examples

### 55. Loop: pick one failure mode; strict vs LLM-judge checks; batch runs because single runs are noisy

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-55
- Quote: (paraphrase)
- Source: Google, three-step loop

### 56. Suite purpose

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-56
- Quote: "did not make the agent holistically worse"
- Source: Google, When to evaluate

### 57. Log context on failed runs and check for the answer

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-57
- Quote: "then check whether the answer was in there"
- Source: Sourcegraph, observability
- URL: https://sourcegraph.com/blog/context-engineering

### 58. Cost scales with what you send

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-58
- Quote: "Cost and latency scale with what you send, not with what you could send"
- Source: Sourcegraph, cost

### 59. Agents ~4x chat tokens, multi-agent ~15x

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-59
- Quote: "agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats"
- Source: Anthropic, 13 Jun 2025, benefits and downsides
- URL: https://www.anthropic.com/research/multi-agent-research-system

### 60. Earn every rule

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-60
- Quote: "Earn every rule"
- Source: Thoughtworks, close
- URL: https://www.thoughtworks.com/insights/blog/architecture/engineering-the-harness-a-practical-pattern-for-reliable-coding-agents

### 61. HumanLayer discards most hooks

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-61
- Quote: "I have thrown away many more hooks than we actually use today"
- Source: HumanLayer, Hooks
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 62. ETH Zurich study: context files (LLM-generated and developer-committed) did not generally improve task success and raised inference cost by over 20%

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-62
- Quote: "does not generally improve task success rates, while increasing inference cost by over 20% on average"
- Source: Gloaguen et al., arXiv 2602.11988, v3 29 Sep 2026, Abstract
- URL: https://arxiv.org/abs/2602.11988

### 63. Repository overviews not helpful; files useful for non-standard practices

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-63
- Quote: "repository overviews, although popular and recommended by model providers, are not helpful"; "useful for specifying non-standard coding practices"
- Source: Gloaguen et al., Abstract
- URL: https://arxiv.org/abs/2602.11988

### 64. Every harness component encodes an assumption about what the model can't do, and these go stale as models improve

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-64
- Quote: "encodes an assumption about what the model can't do on its own"; "can quickly go stale as models improve"
- Source: Anthropic, Harness design for long-running application development, 24 Mar 2026, simplifying the harness
- URL: https://www.anthropic.com/research/harness-design-long-running-apps

### 65. HumanLayer frames harness engineering as a subset of context engineering, via harness configuration points

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-65
- Quote: "the subset of context engineering which primarily involves leveraging harness configuration points"
- Source: HumanLayer, What is harness engineering
- URL: https://www.hlyr.dev/blog/skill-issue-harness-engineering-for-coding-agents

### 66. Tool-result clearing, compaction and memory are offered as API features (cookbook audience = API builders); Claude Code uses compaction and memory

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-66
- Quote: "Our API offers first-party implementations of all three"
- Source: Claude Cookbook, intro
- URL: https://platform.claude.com/cookbook/tool-use-context-engineering-context-engineering-tools

### 67. The comparison table's question row and 'typical failure' row come from SmartScope's context-vs-harness table

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-67
- Quote: (paraphrase: "What to show" vs "What to prevent, measure, control, and fix"; "A single output is inaccurate" vs "Quality degrades over time")
- Source: SmartScope, comparison table
- URL: https://smartscope.blog/en/blog/harness-engineering-overview/

### 68. Lulla et al.: with AGENTS.md, median runtime −28.64%, output tokens −16.58%, comparable task completion (10 repos, 124 PRs)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-68
- Quote: "lower median runtime"; "reduced output token consumption"; "comparable task completion behavior"
- Source: Lulla et al., arXiv 2601.20404, 28 Jan 2026, Abstract
- URL: https://arxiv.org/abs/2601.20404

### 69. Thoughtworks: what the model lacked in the rename case was system-level visibility, guardrails and automated feedback

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-69
- Quote: "system-level visibility, architectural guardrails and automated feedback mechanisms"
- Source: Thoughtworks, opening
- URL: https://www.thoughtworks.com/insights/blog/architecture/engineering-the-harness-a-practical-pattern-for-reliable-coding-agents

### 70. Google runs evals in batches because single runs are noisy

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-70
- Quote: "can be noisy due to nondeterminism of AI models"
- Source: Google Developers Blog, batch evaluations
- URL: https://developers.googleblog.com/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/

### 71. Anthropic's working rule, from Building Effective Agents

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-71
- Quote: "find the simplest solution possible, and only increase complexity when needed"
- Source: Anthropic harness design, simplifying the harness
- URL: https://www.anthropic.com/research/harness-design-long-running-apps

### 72. Anthropic's Tool Search Tool defers tool definitions until needed; ~85% fewer tokens in its example

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-72
- Quote: "85% reduction in token usage"; "defer_loading"
- Source: Anthropic, Introducing advanced tool use, 24 Nov 2025, Tool Search Tool
- URL: https://www.anthropic.com/research/advanced-tool-use

### 73. Claude Code defers MCP tool definitions until needed

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-73
- Quote: "deferring tool definitions until Claude needs them"
- Source: Claude Code docs, MCP, Scale with MCP tool search
- URL: https://code.claude.com/docs/en/mcp

### 74. Claude Code hooks run shell commands with the user's full permissions

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-74
- Quote: "execute shell commands with your full user permissions"
- Source: Claude Code docs, hooks, Security considerations
- URL: https://code.claude.com/docs/en/hooks

### 75. Claude Code deny rules block a tool and are evaluated first

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-75
- Quote: "Deny rules prevent Claude Code from using the specified tool"; "Rules are evaluated in order: deny, then ask, then allow"
- Source: Claude Code docs, permissions, permission rules
- URL: https://code.claude.com/docs/en/permissions

### 76. Claude Code's sandbox limits which directories a command can write to

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-76
- Quote: "restricts file system access to specific directories"
- Source: Claude Code docs, sandboxing, How sandboxing works
- URL: https://code.claude.com/docs/en/sandboxing

### 77. Databricks: episodic memory (raw session traces, searchable) alone 22.3%→35.8%; derived memory 66.1%; both 68.4%; internal benchmark of enterprise tasks

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-77
- Quote: "Episodic memory alone helps moderately: 35.8%, up from 22.3% with no memory"; "lifting pass rate to 66.1%"; "best result, at 68.4%"
- Source: Databricks, What Makes MemCurrent Work
- URL: https://www.databricks.com/blog/memcurrent-designing-persistent-memory-production-agents

### 78. A later long-running agent would see progress and declare the job done

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-78
- Quote: "look around, see that progress had been made, and declare the job done"
- Source: Anthropic long-running, The long-running agent problem
- URL: https://www.anthropic.com/research/effective-harnesses-for-long-running-agents

### 79. ETH measured inference cost; Lulla measured runtime and output tokens (different efficiency measures)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-79
- Quote: "increasing inference cost by over 20% on average"; "lower median runtime"; "reduced output token consumption"
- Source: Gloaguen et al.; Lulla et al., Abstracts
- URL: https://arxiv.org/abs/2602.11988

### 80. Sourcegraph warns a badly-retrieving sub-agent still returns a confident summary

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-80
- Quote: "a sub-agent that retrieved badly still returns a confident summary"
- Source: Sourcegraph, Sub-agent architectures
- URL: https://sourcegraph.com/blog/context-engineering

### 81. An MCP server's list of tools can change after a client has read it (list-changed notification)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-81
- Quote: "When the list of available tools changes, servers that declared the listChanged capability SHOULD send a notification"
- Source: MCP specification 2025-06-18, List Changed Notification
- URL: https://modelcontextprotocol.io/specification/2025-06-18/server/tools

### 82. Google distinguishes end-to-end benchmarks (outcomes) from behavioural evals (discrete actions)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-82
- Quote: "End-to-end benchmarks are the de facto for evaluating model performance"; "a behavioral eval measures discrete, observable actions"
- Source: Google Developers Blog, intro; behavioural evals
- URL: https://developers.googleblog.com/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/

### 83. Vitest accepts a failure message as expect's second argument (used in the code example)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-83
- Quote: "Vitest supports a message as the second argument"
- Source: Vitest docs, expect, expect
- URL: https://vitest.dev/api/expect

### 84. ETH: developer-provided files did slightly better, 2.4 percentage points on average, not significant (p=21%), and significantly outperformed LLM-generated files (p=3.8%)

- Status: VERIFIED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-84
- Quote: "Developer-provided context files improve agent performance by 2.4% on average (p=21%), significantly outperforming LLM-generated ones (p=3.8%)"
- Source: Gloaguen et al. (full paper, §4.2), 4.2 Main results
- URL: https://arxiv.org/html/2602.11988v3

### 85. The guide's contract check fails on the renamed schema with the quoted message and passes on the original

- Status: EXECUTED
- Row link: https://durabilitycurve.com/claims/what-is-harness-engineering/#c-85
- Quote: (run output)
- Source: Our run, 22 Oct 2026, Vitest 5.0.5 + Zod + TypeScript 7.0.2

## Cite

- A claim: name the original source (and its locator) first, then the row it was checked in, e.g. "<source>, <locator>. Checked in: Claim Ledger, "What Is Harness Engineering? How to Build AI Agents That Catch Their Own Mistakes", The Durability Curve, row <id>, https://durabilitycurve.com/claims/what-is-harness-engineering/#c-<id>."
- The essay: Floyd, Harry (2026). What Is Harness Engineering? How to Build AI Agents That Catch Their Own Mistakes. The Durability Curve. https://durabilitycurve.com/blog/what-is-harness-engineering/
- This ledger: Floyd, Harry (2026). Claim Ledger: What Is Harness Engineering? How to Build AI Agents That Catch Their Own Mistakes. The Durability Curve. https://durabilitycurve.com/claims/what-is-harness-engineering/
- Downloads: https://durabilitycurve.com/claims/what-is-harness-engineering.csv · https://durabilitycurve.com/claims/what-is-harness-engineering.json

Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Reuse it, quote it or train on it, with credit to The Durability Curve and a link. Open to AI (robots.txt: search=yes, ai-input=yes, ai-train=yes).