Claim ledger
The Limit Said 10. The Loop Made 500 Calls.
Your limit counts one cycle. The one that runs away is another. Here is how to tell them apart.
34 claims · 21 verified to primary · 12 first-party runs · 3 struck or excluded
What the essay claims
Believed now: "My loop has a limit. I set it to 10. It is bounded."
Believed after: "Whether a limit bounds anything is a question with a fifty-year-old answer, and I can apply it. My limit counts passes through one cycle; the cycle that runs away is a different one; and the guard I would have added next has the same defect one level down."
The delta that does the work: An instrument, not an anecdote. The bench shows the failure, the variant explains it, and the frame predicts where repeat-detection breaks before the reader is shown that it does
The claim ladder
What this ladder does NOT reach:
| # | Claim | Subject — whose behaviour it measures | Rests on | Status / scope |
|---|---|---|---|---|
| 1 | Termination needs a quantity that decreases on every pass through the cycle, in units that run out, and something that tests it | Code structure, any program | Loop variant; Turing, Floyd, Cook et al. | Settled, borrowed, ~50 years old. Conceded at the point of use. The fourth clause is the one agent writing drops |
| 2 | A limit therefore bounds only the cycle whose passes it counts, so a limit can be set, honoured on every pass, and irrelevant | Code structure | Rung 1 applied | The frame's first prediction. Demonstrated, not asserted |
| 3 | Here is that, at 500 calls, runnable with no key | Our bench, n=1, deterministic | First-party run | A demonstration of a borrowed mechanism. 500 is the bench's own backstop and the prose says so at first use. A no-key reproduction is not ours (runcycles.io) |
| 4 | At realistic parse-failure rates the same uncovered cycle costs little, and the exposure is the sustained window where p climbs | 2,000 seeded trials per rate, our simulation | loop_variant_lab.py | Ours as a measurement. A simulation of a control-flow shape, not an observation of any live system |
| 5 | Repeat-hash detection has the same defect one level down: its measure stops decreasing when arguments vary | Code structure, then our bench | Rung 1 applied, then measured | The frame's second prediction, and the reason it is a frame rather than an anecdote. Mechanism and defeat both published (West); the numbers are ours |
| 6 | This shape is in at least one real repository | One repository, LiteRAG, in a surveyed corpus | Hou et al., preprint | An existence proof. Never a frequency |
| Not claimed | Why it is out of reach |
|---|---|
| That the reader's loop is broken | The bench is a scripted fake model and rung 4 is a simulation. Neither measures the reader's code |
| That an uncovered cycle will run away | Rung 4 measures the opposite at low failure rates: median overrun of zero at p=0.02. "Can", never "will" |
| That the frame is new | Fifty years old and credited. Only its application here is ours |
| That any of the guards is ours | West published the hash and its defeat; the SDK ships increment-before-call |
| That this failure is common | 47 of 6,549 is a lower bound of 0.72% with no recall reported. The corpus bounds prevalence in one direction only, and the piece says which |
| That the guards would have stopped the AWS bill | They would not. That cycle ran inside a CloudFormation retry |
| Anything about what a run costs | Pitch 3 |
What would make it wrong
A reader runs the printed listing and it does not print 500
loop_variant_lab.py at other seeds moves the p=0.02 median off 10
Someone has already published loop-variant reasoning as the organising frame for agent loops
A reader shows a terminating loop the four-part question flags as broken
Framework-supplied retries turn out to be unbounded by default
The evidence, row by row
Each row is a claim the essay makes, the words in the source that support it, where in the source they are, the day the source was read, and the status the check assigned. Verified means the primary was opened and read; Executed means we ran it ourselves; Reported means carried from a source we could not open in full.
-
The opening incident ran 9–10 May 2026, not June, and provisioned five m8g.12xlarge instances plus load balancers and Lambdas, duplicated by repeatedly deploying the same CloudFormation template, against DN42.
Timeline: began "2026-05-09", agent shut down roughly 24 hours later on "2026-05-10". Agent's own proposal: "My primary objective is to conduct comprehensive (full port) network scanning and topological data gathering." Five "m8g.12xlarge" instances; operator: "many instance and load balancer and lambda", duplicated by redeploying "the same cloudformation template"
Lan Tian, blog post on the DN42 scanning agent · § timeline + § what it built · lantian.pub
-
The $6,531.30 figure is the bill as incurred; AWS subsequently reduced it to about $1,894
Initial bill "$6531.30"; operator reports the revised amount as "$1894" after AWS agreed to reduce charges
Lan Tian, ibid. · § aftermath · lantian.pub
-
The "a small VPS would have done it" judgement is the blog author's, not the operator's
"for a hobbyist network like DN42, such infrastructure is way overkill, a small VPS server would do the job" — author's own assessment
Lan Tian, ibid. · § assessment · lantian.pub
-
The operator learned of the spend through card charges rather than AWS monitoring
Operator: "the cost too high and much charges on card"
Lan Tian, ibid. · § discovery · lantian.pub
-
AWS Budgets can attach an enforcing action, up to a Deny IAM policy, but budget data refreshes at most three times a day, typically 8–12 hours apart. It fires late, not never, and late is the stronger sentence
"AWS Budgets information is updated up to three times a day. Updates typically occur 8–12 hours after the previous update." · "Apply a custom Deny IAM policy that restricts the ability for a user, group, or role to provision additional Amazon EC2 resources." (second locator budgets-controls.html) · "You might incur additional costs or usage that exceed your budget notification threshold before AWS Budgets can notify you"
AWS Cost Management docs. Console/service setting — not executable, no package to import · § Managing your costs with AWS Budgets — console setting, service-side, nothing to run · docs.aws.amazon.com
-
The quote supports only that On-Demand quotas exist per account per Region and are denominated in vCPUs. It does NOT support "the fast structural control the AWS incident did not have set", and the same page contradicts that framing outright: quotas are automatically increased with usage, so they are never off, only sized. Two independent reads flagged the row and the caller re-verified at source. The published default for the Standard family is 5, quoted below
"There are quotas for the number of running On-Demand Instances per AWS account per Region… managed in terms of the number of virtual central processing units (vCPUs)" ·
AWS, Amazon EC2 User Guide. Console/service setting — not executable · § On-Demand Instance quotas — console setting, service-side. · docs.aws.amazon.com
-
Ball's framing of an agent is "an LLM, a loop, and enough tokens"
"It's an LLM, a loop, and enough tokens. It's what we've been saying on the podcast from the start."
Thorsten Ball, How to Build an Agent · § opening · ampcode.com
-
A working code-editing agent fits in under 400 lines, mostly boilerplate
"You can do it in less than 400 lines of code, most of which is boilerplate."
Ball, ibid. · § opening · ampcode.com
-
The canonical tutorial does not cover error recovery, termination, token budget, cost, tool-output verification, concurrency or streaming
Ball, ibid. · full page, 7 headings · ampcode.com
-
Ball's tutorial does not cover verifying what a tool returned
Absence claim, checked at source against all 7 headings. The only handling of a tool's return is executeTool flipping an error boolean — return anthropic.NewToolResultBlock(id, err.Error(), true) — which captures an error raised BY the tool, not any check on whether what came back is correct or usable. Found by a cold read, not by a gate
Thorsten Ball, How to Build an Agent · full page, 7 headings · ampcode.com
-
Anthropic defines agents as systems where the model directs its own process
agents are "systems where LLMs dynamically direct their own processes and tool usage"
Anthropic, Building Effective Agents · § What are agents? · anthropic.com
-
Anthropic advises agents need ground truth from the environment each step
"During execution, it's crucial for the agents to gain 'ground truth' from the environment at each step"
Anthropic, ibid. · anthropic.com
-
A tool result is returned to the model in a tool_result content block carrying the tool_use_id of the call it answers; the id is what pairs the two halves of the exchange
"Your code executes the operation and sends back a tool_result." · "a second request sends the result back in a tool_result block so Claude can reply with the answer" · request body, verbatim from the worked example: {"type": "tool_result", "tool_use_id": tool_use.id, "content": weather}
Anthropic, Tool use with Claude. Service-side API contract — not executable, no package to import: the requirement is enforced by the API, not by a value readable in the SDK. · § How tool use works — service-side, console/service contract, nothing to run locally · platform.claude.com
-
Every mainstream framework ships a step-counting bound, read today out of each project's current wheel: langchain-classic 1.0.8 AgentExecutor.max_iterations default 15 · langgraph 1.2.11 recursion_limit default 10007 · crewai 1.15.16 max_iter default 25 · openai-agents 0.21.0 max_turns default 10 · autogen-agentchat 0.7.5 composable TerminationCondition classes. Its only occurrence there is a local loop variable in summarization.py (max_iterations = len(messages).bit_length() + 1, line 697). That is false, and this row's own quoted output disproves it: autogen-agentchat 0.7.5 ships eleven concrete TerminationCondition classes and exactly one counts steps (MaxMessageTermination, line 62). TokenUsageTermination (line 235) counts tokens via max_total_token / max_prompt_token / max_completion_token; TimeoutTermination (line 358) counts wall-clock via timeout_seconds; the remaining eight are content or external triggers with no counter.
Run output, python3 Automation/framework_bounds_probe.py, 6/6 probes resolved: "langchain-classic 1.0.8 langchain_classic/agents/agent.py — AgentExecutor.max_iterations default: 15 (line 1023)" · "langgraph 1.2.11 langgraph/_internal/_config.py — recursion_limit default: 10007 (line 32)" · "crewai 1.15.16 crewai/agents/agent_builder/base_agent_executor.py — max_iter default: 25 (line 27)" · "openai-agents 0.21.0 agents/run_config.py — max_turns default: 10 (line 43)" · "autogen-agentchat 0.7.5 autogen_agentchat/conditions/_terminations.py — StopMessageTermination (line 24), MaxMessageTermination (line 62), TextMentionTermination (line 111), FunctionalTermination (line 158), TokenUsageTermination (line 235), HandoffTermination (line 313)" The probe prints six per line; the class total is eleven, of which exactly one counts messages, one counts tokens, one counts wall-clock seconds, and the remaining eight fire on message content or on an external signal (ExternalTermination inspects no message at all; it fires when caller code calls .set()). Independently reproduces C95's 10007 by a different route.
First-party execution against current PyPI wheels · Automation/framework_bounds_probe.py; source locations langgraph/_internal/_config.py line 32, crewai/agents/agent_builder/base_agent_executor.py line 27, agents/run_config.py line 43, langchain_classic/agents/agent.py line 1023
-
Cloudzy describes a recursion limit as a backstop firing after the waste
"It is a backstop that fires _after_ the loop has already wasted twenty-five steps and the API spend that goes with them."
Cloudzy. Verified verbatim. · § recursion limits · cloudzy.com
-
An open author copy exists and the quoted sentence verifies verbatim. The paper does NOT say termination requires a per-cycle measure; that framing is what its contribution denies. It presents the single-ranking-function method as Turing's, and argues past it via disjunctive well-foundedness.
"The problem with Turing's method is that finding a single, or monolithic, ranking function for the whole program is typically difficult, even for simple programs." Re-verified first-party by the caller with pypdf: PDF page 4 = journal page 91. Reference 39 on the last page reads "Turing, A. Checking a large routine. In Report of a Conference on High Speed Automatic Calculating Machines", which corroborates the Turing-1949 attribution
Cook, Podelski & Rybalchenko, CACM 54(5):88–98, May 2011. Open author copy; ACM's own page still 403s · journal p.91 (PDF p.4), § Turing's Classic Method and Disjunctive Well-Foundedness · www0.cs.ucl.ac.uk
-
The open scan at the URL below is a 14-page image PDF with no text layer — pypdf extracts zero characters from all 14 pages, so the absence of the quoted strings proves nothing and their presence cannot be confirmed by this route either. What IS verified, first-party, is the Turing half: Cook et al.'s reference 39 is "Turing, A. Checking a large routine…" and their § heading is "Turing's Classic Method" (see C17).
Floyd's authorship of the well-ordered-set termination argument is standard secondary attribution. No verbatim quote is available to this session
Floyd, Assigning Meanings to Programs (1967), scanned copy · not machine-readable; body must attribute, never quote · people.eecs.berkeley.edu
-
A loop with max_iterations=10 issues 500 model calls when the retry sits inside the bounded iteration; 500 is the bench's own backstop, not a natural stop
Run output: "malformed -> {'stopped': 'runaway', 'calls': 500}". hard_cap=500 is an argument in uncovered_bound_loop. Body must name the backstop at first use in the opening, not only where the code appears
First-party run · loop_lab.py §2 — re-run 2026-08-15 17:53 BST
-
No-progress detection on hashed (tool, args) halts a repeating caller in 3 calls
Run output: "repeater -> {'stopped': 'no-progress', 'calls': 3, 'repeated': 'a099dc8c'}"
First-party run · loop_lab.py §3 — re-run 2026-08-15 17:53 BST
-
That same guard does NOT catch a model making novel calls forever; it runs to the ceiling at 30
Run output: "never_finishes -> {'stopped': 'ceiling', 'calls': 30}"
First-party run. The 30 is max_iterations=30 in the guard's own signature, not a discovered threshold · loop_lab.py §3 — re-run 2026-08-15 17:53 BST
-
The three printed outputs of bounded_run are what the printed code produces
Extracted all fenced blocks from the body and executed each runnable one. never_parses -> {'exit': 'budget', 'calls': 20}, always_repeats -> {'exit': 'stall', 'on': 'search', 'calls': 3}, never_finishes -> {'exit': 'budget', 'calls': 20} — byte-identical to the article's output block. The same check passed for the 500-call runaway bench and both guarded() lines.
First-party run, reproducible
-
A static scan of 6,549 agent repos confirmed 68 infinite-loop failures across 47 projects at 91.9% precision
"On the real-world corpus of 6,549 LLM agent projects, IAL-SCAN reports 74 potential findings. Manual review confirms 68 IAL failures and 6 false positives, yielding an end-to-end precision of 91.9%." … "These failures affect 47 agent projects"
Hou, Wang, Zhao & Wang (HUST). PREPRINT. Precision only — no recall reported, so no prevalence inference is available in either direction · arXiv:2607.01641v1 §V-B1 · arxiv.org
-
All 68 confirmed failures share one root cause: the repeating path is not covered by a strong bound
"All 68 failures share the same root issue: the repeated path is not covered by a strong bound."
Hou et al. PREPRINT · arXiv:2607.01641v1 §V-B1 · arxiv.org
-
The bench's failure shape occurs in a real repository: a planner nesting two while not success loops around an LLM call and swallowing the parse exception with pass
"Figure 6 shows a retry loop in 2456868764/LiteRAG. The planner uses nested while not success loops to repeatedly request a plan from the LLM. The costly call appears at lines 8–10, where self.llm.invoke(...) is executed until parsing succeeds. However, parser failures are swallowed at lines 12–13". Figure 6 source, extracted verbatim: success = False / while not success: / while not success: / try: / plan = self.llm.invoke(...) / success = True / except OutputParserException: / pass.
Hou et al. PREPRINT · arXiv:2607.01641v1 §V-B2 Case Study + Figure 6 · arxiv.org
-
The preprint's self-assessment figures: the first two authors agreed on 94.6% of the 74 potential findings and settled the rest by discussion; a separate false-negative review confirmed 7 misses
"During independent labeling, the first two authors agreed on 94.6% of the 74 potential findings; the remaining cases were resolved through discussion." · "…Two authors independently labeled them with 92.1% agreement, and all disagreements were resolved with a third author. This process confirmed 7 false negatives of IAL-SCAN."
Hou et al. PREPRINT, extracted with pypdf from the PDF, not a summariser · arXiv:2607.01641v1 §V-B1 and the false-negative paragraph · arxiv.org
-
The uncovered inner cycle costs almost nothing at realistic parse-failure rates and only becomes expensive as p climbs. max_iterations=10 throughout, so 10 calls means the bound held
Run output, python3 Automation/loop_variant_lab.py, 2,000 seeded trials per row: FULL printed table, all fifteen cells ( The lab sweeps p as a chosen parameter and measures nothing about which rate occurs in production.
First-party, seeded and deterministic · Automation/loop_variant_lab.py, run_once() line 38
-
Call-signature hashing with sorted keys, AND its defeat by semantically-equivalent varied arguments, both shipped with working code
"Stable hash of the call signature (sort keys for determinism)" · "Sometimes the agent doesn't repeat the exact same call. It does search("python async"), then search("async in python"), then search("python asyncio"). Same intent, different arguments." — the article then goes further than we do, to embedding-based semantic detection. Verified at source by the caller. No inference required
Alan West, How to Stop Your LLM Agent From Looping Itself Into Oblivion, DEV · § the tracker + § varied arguments · dev.to
-
A runnable no-key simulated runaway predates ours by five months
"The LLM calls in this demo are simulated. No API key is required. The budget enforcement is real." · "the agent loops for 30 seconds — ~595 calls, ~$5.95 — before a safety timeout kills it." Verified at source by the caller. Its structure is a quality-threshold loop rather than an uncovered retry path (
Albert Mavashev, Stop a Runaway AI Agent in Three Lines, runcycles.io, 2026-03-26 · § the unguarded run · runcycles.io
-
The article's own guard.py prints c740a98e; C22 records a099dc8c, which is loop_lab.py's different call shape.
Run output from the article's printed listing: repeater -> {'stopped': 'no-progress', 'calls': 3, 'repeated': 'c740a98e'}. Stable across CPython 3.9.6 and 3.12.12 (sha1(json.dumps(call, sort_keys=True)) is deterministic and default separators have not moved in 3.x), so it will match on the reader's machine
First-party run of the printed listing
-
Its counter is on calls, not on time, and neither tools... nor model(messages) carries a timeout
Executed by the caller with a tool that never returns: hanging tool -> STILL RUNNING after 3s: bounded_run did NOT terminate (SIGALRM at 3s).
First-party execution · the printed bounded_run listing, driven by a blocking tool
-
The OpenAI Agents SDK advances its turn counter and tests the bound before invoking the model
Read out of the current wheel by the caller: openai-agents 0.21.0, agents/run.py line 1296 current_turn += 1 immediately followed by line 1297 if max_turns is not None and current_turn > max_turns:. Both precede the model invocation
First-party, read from the package · agents/run.py lines 1296–1297, openai-agents 0.21.0
-
Ten successful parses at independent parse-failure rate p cost 10/(1-p) model calls on average, so the simulation corroborates a general law rather than standing on five chosen rates
Derivation: calls-per-successful-parse is geometric with mean 1/(1-p); ten of them is negative-binomial with mean 10/(1-p). Checked against an independent 20,000-trial simulation written for this row (different generator and seed from loop_variant_lab.py): means 10.2 / 11.1 / 20.0 / 99.9 / 1001.1 against closed-form 10.2 / 11.1 / 20.0 / 100.0 / 1000.0, and medians 10 / 11 / 20 / 97 / 970 reproducing the article's published 10 / 11 / 20 / 97 / 969. The table the article already printed is confirmed by a route that does not use the article's own lab
First-party derivation, independently re-simulated · body § How often is "forever"?
-
Total calls to reach ten successful parses at failure rate p is negative binomial with r=10 and q=1-p; the article's two columns are that distribution's exact median and 95th percentile, taken from the CDF
Computed from the exact PMF C(n-1, r-1) q^r p^(n-r) accumulated to the quantile. p=0.02 → median 10, p95 11 · p=0.10 → 11, 13 · p=0.50 → 19, 28 · p=0.90 → 97, 154 · p=0.99 → 967, 1568. C101's 2,000-trial simulation agrees within two counts on every row (its medians 10/11/20/97/969, p95 11/13/28/154/1569), which converts it from the evidence into corroboration by a second method. A sample maximum is not a bound, and printing one in a piece whose thesis is precision about what bounds a loop invited exactly the misreading the piece exists to correct
First-party derivation · body § How often is "forever"? + Automation/gen_fullheight_ratechart.py
-
Flattening two cycles into one is not necessary for termination. A nested retry loop terminates just as well when the inner cycle tests the shared budget on its own path. The necessary property is that every path reaching another model call spends from the same budget before it gets there, and that the test reading that budget ends the loop
Executed by the caller: the article's nest preserved, with if calls >= max_calls: return inside the inner while, returns {'exit': 'budget', 'calls': 20} — the same 20 as the flattened bounded_run.
First-party execution · body § The exit test and the counter on the same cycle
Struck or excluded before publication
Claims that were drafted, checked, and removed. They are not claims the essay makes; they are the ones it stopped making, with the reason.
-
This is not an exotic bugunsupported. 47 of 6,549 repos is 0.72%, and C8 reports precision with no recall, so the corpus bounds prevalence in neither direction
-
Forty lines gets you an agent / the forty lines aboverefuted inside the article. Ball says under 400 (C2); the code block is 13 lines. Appeared in the subtitle, a heading and the body
-
Every tutorial in the genre leaves the same three holes / trim is the default in most codebases / most people fail the audit on the first pathunmeasured population claims presented as observation. Each asserts a distribution over developers' code or the literature with nothing behind it
Cite this
- A single claim
"[claim text]" (Floyd, Harry, 2026, https://durabilitycurve.com/claims/the-limit-said-10/)Replace the bracket with the row's claim text. The page URL carries the source; the row's own source link is in the row.- The essay
Floyd, Harry (2026). The Limit Said 10. The Loop Made 500 Calls.. The Durability Curve. https://durabilitycurve.com/blog/the-limit-said-10/- This ledger
Floyd, Harry (2026). Claim Ledger: The Limit Said 10. The Loop Made 500 Calls. [structured claims with sources]. The Durability Curve. https://durabilitycurve.com/claims/the-limit-said-10/
Quote with attribution and a link to this page or the essay. Say if you changed the wording. Not licensed for model training. Plain-text copy for machines: /md/claims/the-limit-said-10.md.