Claim ledger

Same Model, Different Product: The Case for Harness Engineering

Hold the model fixed and the surrounding harness can still move task success, cost and reliability. Here are the layers that do it.

Read the essay On Substack Published Tests Law IV: Instruments Over Theory

Claims
14
Verified at the primary source
14
Removed before publication
0
Last checked

14 claims: 14 verified.

Download this ledger: CSV · JSON · Markdown · CC BY 4.0 · Query it from Claude or ChatGPT

The evidence, row by row

Each row is a claim the essay makes, then the evidence recorded for it and the source it was checked against. For verified rows that is usually the exact words, where they sit and the day they were read. The label says how far it was checked.

What the labels mean
Verified
Checked against the primary source itself by a checker who did not draft the piece, usually with the exact words, where they sit and the date read.
Executed
A run or observation done for the piece: code, a query, a count, or a check made live in an app. The claim is what it returned. Run records are not published.
Checked
Checked against its source by a separate checker in the older ledger format, which recorded no quote, locator or date.
Reported
Not verified word for word against a primary source: carried from a secondary source, from one that could not be opened in full, or supported only in part. The essay words it accordingly.
Struck
Drafted, checked and removed before publication. Kept here, crossed out, with the reason.
Excluded
Considered and deliberately left out of the piece. Kept here with the reason.
Sources (8)
  1. Lee 2 rows
  2. LangChain 1 row
  3. Claude Code Docs 3 rows
  4. Yichao 'Peak' Ji 2 rows
  5. Can Bölük 2 rows
  6. Menlo Ventures 1 row
  7. Anthropic 1 row
  8. DeepSeek AI 2 rows
  1. HE1 Verified Read

    In the Meta-Harness paper's TerminalBench-2 table, Claude Opus 4.6 has pass rates of 58.0 under Claude Code (an official-leaderboard result the paper reports) and 76.4 under the Meta-Harness harness. Scope: Meta-Harness searched and was finally evaluated on the same 89-task TerminalBench-2 benchmark, and the paper describes the resulting harness as specialised to the TerminalBench-2 regime.

    search and final evaluation on the same 89-task benchmark … specialized to the TerminalBench-2 regime … Claude Opus 4.6 … Claude Code × 58.0 … Meta-Harness ✓ 76.4

    Lee, Nair, Zhang, Lee, Khattab and Finn, Meta-Harness: End-to-End Optimization of Model Harnesses, arXiv:2603.28052v1 · p. 9, §4.3 Evaluating Agentic Coding Harnesses on TerminalBench-2 (setup paragraph) · Table 7, Claude Opus 4.6 block; caption: results of others are from the official leaderboard · arxiv.org

    Link
    Cite

    Lee, Nair, Zhang, Lee, Khattab and Finn, Meta-Harness: End-to-End Optimization of Model Harnesses, arXiv:2603.28052v1, p. 9, §4.3 Evaluating Agentic Coding Harnesses on TerminalBench-2 (setup paragraph) · Table 7, Claude Opus 4.6 block; caption: results of others are from the official leaderboard, https://arxiv.org/pdf/2603.28052. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE1, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE1 (source read 2026-10-03).

    Report an error
  2. HE2 Verified Read

    In the same table, Claude Haiku 4.5 has pass rates of 27.5 under Claude Code (an official-leaderboard result the paper reports) and 37.6 under the Meta-Harness harness. Scope: Meta-Harness searched and was finally evaluated on the same 89-task TerminalBench-2 benchmark, and the paper describes the resulting harness as specialised to the TerminalBench-2 regime.

    search and final evaluation on the same 89-task benchmark … specialized to the TerminalBench-2 regime … Claude Haiku 4.5 … Claude Code × 27.5 … Meta-Harness ✓ 37.6

    Lee, Nair, Zhang, Lee, Khattab and Finn, Meta-Harness: End-to-End Optimization of Model Harnesses, arXiv:2603.28052v1 · p. 9, §4.3 (setup paragraph) · Table 7, Claude Haiku 4.5 block; also Figure 1 (right), p. 1 · arxiv.org

    Link
    Cite

    Lee, Nair, Zhang, Lee, Khattab and Finn, Meta-Harness: End-to-End Optimization of Model Harnesses, arXiv:2603.28052v1, p. 9, §4.3 (setup paragraph) · Table 7, Claude Haiku 4.5 block; also Figure 1 (right), p. 1, https://arxiv.org/pdf/2603.28052. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE2, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE2 (source read 2026-10-03).

    Report an error
  3. HE3 Verified Read

    LangChain reports improving its coding agent deepagents-cli by 13.7 points on Terminal Bench 2.0, from 52.8 to 66.5, by changing only the harness while keeping the model fixed at GPT-5.2-Codex.

    improve deepagents-cli … 13.7 points from 52.8 to 66.5 on Terminal Bench 2.0. We only tweaked the harness and kept the model fixed, gpt-5.2-codex

    LangChain (Vivek Trivedy), Improving Deep Agents with harness engineering, 17 February 2026 · opening section, paragraph beginning "We used a simple recipe" · langchain.com

    Link
    Cite

    LangChain (Vivek Trivedy), Improving Deep Agents with harness engineering, 17 February 2026, opening section, paragraph beginning "We used a simple recipe", https://www.langchain.com/blog/improving-deep-agents-with-harness-engineering. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE3, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE3 (source read 2026-10-03).

    Report an error
  4. HE4 Verified Read

    Claude Code's command reference documents /compact as a command that frees up context by summarising the conversation so far.

    /compact [instructions] … Free up context by summarizing the conversation so far.

    Claude Code Docs, Commands · command table, row "/compact [instructions]" · code.claude.com

    Link
    Cite

    Claude Code Docs, Commands, command table, row "/compact [instructions]", https://code.claude.com/docs/en/commands. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE4, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE4 (source read 2026-10-03).

    Report an error
  5. HE5 Verified Read

    Claude Code's environment-variable reference documents CLAUDE_CODE_MAX_RETRIES, which overrides the number of times Claude Code retries failed API requests.

    CLAUDE_CODE_MAX_RETRIES … Override the number of times to retry failed API requests

    Claude Code Docs, Environment variables · Variables list, row "CLAUDE_CODE_MAX_RETRIES" · code.claude.com

    Link
    Cite

    Claude Code Docs, Environment variables, Variables list, row "CLAUDE_CODE_MAX_RETRIES", https://code.claude.com/docs/en/env-vars. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE5, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE5 (source read 2026-10-03).

    Report an error
  6. HE6 Verified Read

    Claude Code's features overview describes subagents as isolated workers with their own context, running separately from the main conversation.

    Subagents … are isolated workers that run separately from your main conversation … Isolated worker with its own context

    Claude Code Docs, Extend Claude Code (features overview) · Compare similar features, tab "Skill vs Subagent", row "What it is" · code.claude.com

    Link
    Cite

    Claude Code Docs, Extend Claude Code (features overview), Compare similar features, tab "Skill vs Subagent", row "What it is", https://code.claude.com/docs/en/features-overview. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE6, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE6 (source read 2026-10-03).

    Report an error
  7. HE7 Verified Read

    Manus warns, from its own experience building an agent rather than from a measured experiment, that when users plug hundreds of tools into an agent's action space, the model is more likely to select the wrong action or take an inefficient path.

    hundreds of mysterious tools … the model is more likely to select the wrong action or take an inefficient path

    Yichao 'Peak' Ji, Manus, Context Engineering for AI Agents: Lessons from Building Manus, 18 July 2025 · section "Mask, Don't Remove", first paragraph · manus.im

    Link
    Cite

    Yichao 'Peak' Ji, Manus, Context Engineering for AI Agents: Lessons from Building Manus, 18 July 2025, section "Mask, Don't Remove", first paragraph, https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE7, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE7 (source read 2026-10-03).

    Report an error
  8. HE8 Verified Read

    Manus deliberately leaves failed actions and errors in the agent's context, arguing that erasing failure removes the evidence the model needs to adapt.

    Erasing failure removes evidence. And without evidence, the model can't adapt. … leave the wrong turns in the context

    Yichao 'Peak' Ji, Manus, Context Engineering for AI Agents: Lessons from Building Manus, 18 July 2025 · section "Keep the Wrong Stuff In", second and third paragraphs · manus.im

    Link
    Cite

    Yichao 'Peak' Ji, Manus, Context Engineering for AI Agents: Lessons from Building Manus, 18 July 2025, section "Keep the Wrong Stuff In", second and third paragraphs, https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE8, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE8 (source read 2026-10-03).

    Report an error
  9. HE9 Verified Read

    In Can Bölük's edit-format benchmark, the hashline edit format beat the patch format in 14 of 16 models.

    hashline beats patch in 14/16 models

    Can Bölük, Stencil, We improved 15 LLMs at coding in one afternoon. Only the harness changed., 12 February 2026 · benchmark chart caption (3 runs × 180 tasks per model) · stencil.so

    Link
    Cite

    Can Bölük, Stencil, We improved 15 LLMs at coding in one afternoon. Only the harness changed., 12 February 2026, benchmark chart caption (3 runs × 180 tasks per model), https://stencil.so/blog/the-harness-problem. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE9, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE9 (source read 2026-10-03).

    Report an error
  10. HE10 Verified Read

    In the same benchmark, Grok Code Fast 1 went from 6.7% with the patch edit format to 68.3% with hashline.

    Grok Code Fast 1 went from 6.7% to 68.3%, a tenfold improvement … patch 6.7% … hashline 68.3%

    Can Bölük, Stencil, We improved 15 LLMs at coding in one afternoon. Only the harness changed., 12 February 2026 · results paragraph beginning "Sixteen models, three edit tools" · chart row 01 point labels · stencil.so

    Link
    Cite

    Can Bölük, Stencil, We improved 15 LLMs at coding in one afternoon. Only the harness changed., 12 February 2026, results paragraph beginning "Sixteen models, three edit tools" · chart row 01 point labels, https://stencil.so/blog/the-harness-problem. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE10, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE10 (source read 2026-10-03).

    Report an error
  11. HE11 Verified Read

    Menlo Ventures estimated in December 2025 that Anthropic held 54% of the enterprise coding market, up from 42% six months earlier, driven in large part by Claude Code. The figure comes from the report's LLM market-share analysis, which estimates model-provider share from production API usage.

    coding market … an estimated 54% market share … up from 42% just six months ago, driven in large part by the popularity of Claude Code

    Menlo Ventures, 2025: The State of Generative AI in the Enterprise, 9 December 2025 · section "LLM Market Share: Anthropic Extends Its Lead in the Enterprise", paragraph after the share charts · methodology note "LLM Market Share" · menlovc.com

    Link
    Cite

    Menlo Ventures, 2025: The State of Generative AI in the Enterprise, 9 December 2025, section "LLM Market Share: Anthropic Extends Its Lead in the Enterprise", paragraph after the share charts · methodology note "LLM Market Share", https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE11, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE11 (source read 2026-10-03).

    Report an error
  12. HE12 Verified Read

    Anthropic said in February 2026 that Claude Code's run-rate revenue had grown to over $2.5 billion.

    Today, Claude Code’s run-rate revenue has grown to over $2.5 billion

    Anthropic, Series G funding announcement, 12 February 2026 · paragraph beginning "Claude Code represents a new era of agentic coding" · anthropic.com

    Link
    Cite

    Anthropic, Series G funding announcement, 12 February 2026, paragraph beginning "Claude Code represents a new era of agentic coding", https://www.anthropic.com/news/anthropic-raises-30-billion-series-g-funding-380-billion-post-money-valuation. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE12, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE12 (source read 2026-10-03).

    Report an error
  13. HE13 Verified Read

    DeepSeek publishes DeepSeek Harness (dsh) from its deepseek-ai GitHub organisation, and the README describes it as an open-source agent harness developed by DeepSeek AI.

    DeepSeek Harness (dsh) is an open-source agent harness developed by DeepSeek AI.

    DeepSeek AI, deepseek-ai/deepseek-harness README, GitHub · README opening paragraph · DeepSeek's own site, deepseek.com/harness, links the repository · github.com

    Link
    Cite

    DeepSeek AI, deepseek-ai/deepseek-harness README, GitHub, README opening paragraph · DeepSeek's own site, deepseek.com/harness, links the repository, https://github.com/deepseek-ai/deepseek-harness. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE13, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE13 (source read 2026-10-03).

    Report an error
  14. HE14 Verified Read

    The DeepSeek Harness README labels the project a developer preview.

    DeepSeek Harness is in developer preview and iterating rapidly. THERE WILL BE COMPATIBILITY-BREAKING CHANGES.

    DeepSeek AI, deepseek-ai/deepseek-harness README, GitHub · README section "Developer preview" · github.com

    Link
    Cite

    DeepSeek AI, deepseek-ai/deepseek-harness README, GitHub, README section "Developer preview", https://github.com/deepseek-ai/deepseek-harness. Checked in: Claim Ledger, "Same Model, Different Product: The Case for Harness Engineering", The Durability Curve, row HE14, https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/#c-HE14 (source read 2026-10-03).

    Report an error

Cite and reuse

A single claim
Use the row's Cite button. It names the original source first, then this row, which carries its own link.
This ledger
Floyd, Harry (2026). Claim Ledger: Same Model, Different Product: The Case for Harness Engineering. The Durability Curve. https://durabilitycurve.com/claims/harness-engineering-same-model-different-product/
The essay
Floyd, Harry (2026). Same Model, Different Product: The Case for Harness Engineering. The Durability Curve. https://durabilitycurve.com/blog/harness-engineering-same-model-different-product/

The ledger data is licensed CC BY 4.0: reuse it, quote it or train on it, with credit to The Durability Curve and a link. Read the licence. Machine copies: JSON, CSV, Markdown.