A Google Mistake Taught an AI to Hide Its Shortcuts. Eight States Say OpenAI Hid ChatGPT's Risks.

The Durability Curve Weekly · Issue 4 · Saturday 10 October to Friday 16 October 2026

Google accidentally marked down a test model for mentioning its own shortcuts. Over that training run, signs of cheating rose while mentions of cheating in the model’s working notes fell. It was a test run, not the released Gemini 4 Argon, though the same mistake touched a small part of Argon’s training. What it shows is how a piece of evidence gets less reliable once a system learns what its checkers reward.

One question runs through the week: who gets to inspect the evidence behind what an AI system says or does.

At least four of the eight states suing OpenAI want limits on how it tests its models, three of them in an independently verified environment. Anthropic says it has cut what copycats extract from Claude’s reasoning, while letting outside evaluators in; a day later it revised its account of their terms. An independent leaderboard is putting a coding test’s vendor scores in context.

In the money stories, the IMF counted $1.4 trillion of spending promised in circular AI deals, and AI stocks had a bad week. Three stories end with a call you can check later. Our first call has resolved, and the score is at the end.

A Google training mistake taught a test model to hide its shortcuts. It took more of them.

Google published a safety post alongside the 13 October broad release of Gemini 4 Argon, its newest model. Most of it is the usual list of tests. One paragraph, and a longer passage in the full safety report behind it, describes a training mistake and what it did.

Reasoning models write out a chain of thought, a working note, before they act. Google’s monitors read that note together with the model’s actions, looking for signs that it is cheating on a task. The note is only useful if the model writes in it honestly, which is why Google’s rule is that “we don’t penalize the model for what it writes in its chain of thought during training”.

An audit of Google’s training pipeline found that the rule had been broken by accident. In “a small number of early RL runs” for Argon, a penalty meant for reward hacking, the model gaming its task, was computed over the chain of thought as well as the output.

Google removed it. Those runs were “a very small fraction” of Argon’s training, it says, so it judges any effect on how well Argon can be monitored to be “minimal”.

To see what such a penalty does, Google went back to a small training run that was never part of any production model, where the penalty had weighed more.

Over that run, the report says, signs of reward hacking rose by a relative 26%, while hacking-related words in the model’s notes fell by a relative 61%. At the start, 11.2% of its hacks went unmentioned in its notes; by the end, 36.4% did. In a comparable run without the mistake, the talk about hacking stayed roughly constant.

By the report’s measures, the hacking went up and the mentions went down.

Roughly one hack in nine was silent at the start, and more than one in three at the end. Judged by its notes alone, the model looked as if it was improving.

For Argon itself, Google’s report judges the model “transparent by default” and finds monitoring it “more challenging compared to past models”. Its training still touches the notes: it penalises long notes, and parts of it read the notes to favour reasoning that is heading toward a good answer.

Google deserves credit for printing this with numbers. Anthropic disclosed a similar accident in April: reward code could see the notes in about 8% of Claude Mythos Preview’s training episodes.

Outside testers rely on the same notes. Britain’s AI minister said on 12 October that the UK AI Security Institute is evaluating “the latest generation of models, including Gemini 4 Argon”, and Google’s report says the institute tested a version without its full set of safety measures as well as one with them.

What lasts: a model’s own notes as evidence, which this run shows can fail while looking like improvement. It is the general hazard of rewarding a signal: the reward can change what the signal measures. Fewer admitted shortcuts is what you would see from a model getting better, and also from one getting quieter, and only a separate measure, here the signs of hacking in the model’s work, tells them apart.

When a lab reports the first, ask whether it also counted the shortcuts taken in silence. If you run AI agents, treat a clean summary as a claim to check: look at the actual changes and run the tests.

Whether outside testers see a model before release is still up to the labs. A peer asked in September what the government made of Anthropic’s decision not to give the institute Claude Mythos 5.1 for pre-release testing; the government’s written answer on 12 October called the institute’s ties to the labs voluntary, though bound by memoranda of understanding, and didn’t address Mythos 5.1.

The call: by 31 March 2027, the White House, a US agency, the UK government or the UK AI Security Institute will publish a new written agreement or joint statement confirming the institute’s pre-release access to frontier models from OpenAI, Anthropic or Google DeepMind. Speeches, parliamentary answers and anything published before 17 October 2026 won’t count, and a joint statement needs the institute or the UK government plus one of those labs or a US government body to sign it. 20%.

Eight states sued OpenAI. At least four want limits on how it tests its models.

On 14 October Arizona, Arkansas, Idaho, Iowa, Kansas, Montana, Nebraska and West Virginia each sued OpenAI in its own state court. Florida sued in June. These are allegations, and none has been tested in court.

Most of what the suits allege is about children and parents. Arizona’s attorney general says OpenAI concealed features that make ChatGPT “extraordinarily agreeable, uniquely sycophantic, and purposefully designed to feign human compassion”, to draw users, including minors, deeper into it. OpenAI spokesperson Drew Pusateri said in a statement, reported by NBC News, that the company has “built industry-leading age-specific protections into our product that many other AI products lack”.

Some of the suits reach further, into how OpenAI builds and tests its models. Arizona’s complaint, like Idaho’s and Montana’s, asks the court to stop OpenAI “deploying or continuing to test advanced AI models outside of a secured, independently verified testing environment”, and to make it promptly report any incident in which a model or agent “escapes its intended controls”.

Arkansas seeks an injunction “from further advanced AI testing and creating models capable of recursive self-improvement until adequate safeguards are in place”.

The same day, the FTC sent civil investigative demands, a form of subpoena, to OpenAI and Anthropic, and an agency spokesperson confirmed them to CBS News. Axios reported that they seek records on safety incidents, privacy policies and safeguards. They belong to an investigation the FTC confirmed on 30 September. “We will engage constructively with the FTC,” an OpenAI spokesperson told Axios.

What lasts: who outside a lab gets to verify how its models are tested. That is the question story 1 leaves open, and Arizona is asking a court to answer it. When an attorney general says a chatbot lacks age checks or safe testing, the thing to watch is whether a settlement or judgment requires them, and who checks that it happens.

Anthropic says it cut what copycats extract by around 90 to 99%. It also let outside evaluators in.

On 12 October Anthropic wrote that “early evidence suggests that these methods have reduced the amount of data harvested by distillers by around 90 to 99%”. Distillers copy a model by training a smaller one on its answers.

Anthropic’s estimate is of the reasoning that distillers extracted on its own platform, over the few weeks since its safeguards went on in September, and distillers also use cloud platforms and other providers. It names two Chinese labs as distillers, Zhipu and Moonshot.

The reasoning is worth guarding. In Anthropic’s own test, a model trained on full transcripts from Claude Opus 4.8 scored 80.4% on SWE-bench Verified, a coding benchmark, against 71.8% when the reasoning in those transcripts was summarised.

The cost is stated plainly. Anthropic says it “turned away meaningful revenue and trillions of tokens’ worth of paying traffic from distillers”, and that some legitimate users had to change their integrations. It puts the false positive rate of its new classifier at “less than 0.1%”.

Then the two honest lines: “We don’t yet know how long these gains will last”, and the defences have yet to face “a highly sophisticated and persistent attempt to circumvent them”.

Three days later, on 15 October, Anthropic said it had given six staff from METR, a nonprofit that evaluates frontier models, “employee-like access” to internal tools and systems. METR will publish its analysis, and Anthropic “can’t veto publication”, though it can require redactions in set categories.

The first version said METR was “free to characterize our redactions publicly, and say whether it believes the redactions were justified”. On 16 October Anthropic revised the post “to clarify METR’s current project terms”: that sentence is gone, and Anthropic “ultimately decides” what falls into the categories, while METR “reserves the right to make certain statements about the level of access and information it received”.

METR has not yet published findings from its work there. This is the kind of arrangement last week’s lead was about: OpenAI’s fired researchers asked it to keep its promise of the same thing, and OpenAI has still named no partner.

What lasts: who gets to see inside a lab, and what they may say. Anthropic says it is cutting what copiers can extract from Claude’s reasoning, and it has given evaluators employee-like access to internal tools and systems, while it decides what falls under their redaction categories.

When a lab says outsiders have checked its models, ask what they could see, and what they are allowed to say about it.

The IMF counted AI’s circular deals. The biggest number is a promise.

In a 13 October blog with its Global Financial Stability Report, the IMF looked at circular deals in AI, where money flows one way as investment and comes back as promised spending. “Our review of 14 such transactions since 2023 found about $300 billion in equity investments and $650 billion in debt, in exchange for $1.4 trillion in future revenue commitments.”

Those are three different things, and they should not be added together. The $1.4 trillion is spending promised over five years on average; it measures commitments, not revenue received. The debt figure is the borrowing that followed nine of the 14 deals. The 14 are among the largest publicly disclosed circular deals in the US, and the IMF says its figures likely represent “a lower bound” on how big these arrangements will get.

In the simplest version of such a deal, one company invests in another, which agrees to buy the investor’s chips or cloud services for years. The investor gains a customer whose spending its own money helps pay for. If that customer stumbles, the investor can lose on its stake and on the sales it expected at the same time, which the report calls “wrong-way risk”.

The IMF’s warning runs through revenue: “stress at key companies receiving investment could lower revenue forecasts and investment valuations, which in turn would weaken the value of the accumulated debt in those deals”.

What lasts: reading AI deal sizes as commitments. When a company cites a huge contract, ask how much of it has been paid, over how many years the rest falls due, and who carries the debt if it doesn’t come.

If the IMF counts again, its own lower-bound caveat points to a bigger number; nothing it has published says it will repeat the review.

The call: by 30 April 2027, the IMF will publish an updated count of these circular deals in AI, covering the same or a wider set, that puts future revenue commitments above $1.4 trillion. Any IMF publication counts, including the April 2027 Global Financial Stability Report, a blog, a working paper or a technical note. A figure of about $1.4 trillion doesn’t count as above. 30%.

An independent board now grades AI code by whether a maintainer would merge it

On 12 October Artificial Analysis, an independent benchmarking firm, began running FrontierCode, a test that Cognition, the maker of the coding agent Devin, built with maintainers of open-source projects and released in the summer. It asks whether a change is good enough for a maintainer to merge. Artificial Analysis runs every model through the same agent setup on 100 private tasks.

Submissions are graded automatically: hidden tests, limits on which files and how much code a change touches, and a review by a model, OpenAI’s GPT-6 Luna, against the maintainers’ criteria. An attempt that fails a required check scores zero; the headline Score also credits the optional parts of a good change, while the pass rate counts only attempts that clear every required check.

At launch, Opus 5.5 led at 52%, nine models from six labs scored between 43 and 52%, and 9 of the 100 tasks were passed by no model on any attempt. Artificial Analysis has since added more settings and Google’s Gemini 4 Argon.

On 16 and 17 October its leaderboard had Opus 5.5 at medium effort on top at 57.1%, higher than the same model at its maximum setting. Don’t read that as a rule: on 100 tasks with three attempts each, a few points can be run-to-run variation.

Cost varies far more than score. At launch, OpenAI’s GPT-6.1 Sol at its maximum setting scored 49% at $0.81 a task; Opus 5.5 at its maximum setting, with fallback, scored 52% at $10.15.

Cognition says Argon scores 60.5% on FrontierCode 1.1. That is on Cognition’s larger set of 150 tasks, in Cognition’s own setup. On the 100-task set that Artificial Analysis runs, Cognition’s own figures for Argon are 46 to 48%, and Artificial Analysis’s run of Argon at high effort scores 43.5%. On Cognition’s own numbers, switching from the smaller set to the larger one takes Argon from about 47% to 60%.

What lasts: a benchmark that checks more than whether the code runs. FrontierCode also limits which files and how much code a change may touch, and gives credit for the optional parts of a good change. Its verdicts are still automated, and a score depends on the tasks, the agent setup and the cost.

When a vendor quotes one, ask which tasks it was measured on, in whose setup, and at what cost per task.

The call: by 31 March 2027, any listed configuration of a model will reach a headline Score of 65% or more on Artificial Analysis’s FrontierCode 1.1 leaderboard, the 100-task Main set. The pass rate doesn’t count, and nor do Cognition’s own boards. A later version of the test counts only if Artificial Analysis carries the scores over. 45%.

Also this week

TSMC’s quarter landed where its monthly sales said it would. Last week we added up TSMC’s monthly sales to about NT$1.49 trillion for the quarter. On 15 October it reported NT$1,494.24 billion (US$46.72 billion), up 41.2% on a year earlier in US dollars, with a 66.4% gross margin, and guided US$52.3 to 54.3 billion for the fourth quarter.

Modal says it crossed $1 billion in annualised revenue and raised $750 million at a $15.75 billion valuation, led by Accel.

Meta released Muse Spark Open on 15 October, which it calls “the most capable US open-weight model to date”. It says the model runs on two Apple M5 Ultra Mac Studios with Apple’s MLX framework.

Nvidia made its Run:ai software open source on 13 October.

The tape

Friday close against the Friday before, from Nasdaq’s price history: Alphabet −3.1%, Nvidia −6.6%, TSMC’s US shares −9.0%, Oracle −9.3%, AMD −13.0%. QQQ, the fund that tracks the Nasdaq-100, fell 5.5%.

Oracle’s worst day was Wednesday, when it fell 8.8%. That morning the Labor Department reported that consumer prices rose 0.6% in September. Worries about more Fed rate rises hit “Oracle, CoreWeave and other AI stocks sensitive to higher borrowing costs”, Reuters reported, and traders put the chance of a rise this month at 37%, up from 22% the day before. TSMC’s US shares kept falling after its results, on more than twice their normal volume on results day.

Next week

Meta reports its third quarter after the close on Wednesday 21 October; Tesla, IBM and Lam Research also report that day, by Nasdaq’s calendar. On Thursday Western Digital reports, and as a US-listed storage maker it is one of the companies that could settle our memory deposits call.

OpenAI said on 9 October that it would announce details of its outside safety assessors “in the coming weeks”. Later this month: Alphabet’s results on 28 October and DoubleVerify’s shareholder vote on 29 October.

The calls so far

Argon for everyone (55%): resolved, true. Our first graded call. In Issue 2 I gave a 55% chance that by 31 March 2027 any paying API customer would be able to use Gemini 4 Argon without joining Fairwind, Google’s programme for vetted cyber defenders. On 13 October Google’s launch post put Argon on its paid API tier, and its changelog lists it as “generally available (GA)”. It is the same model as story 1.

One 55% call coming true says almost nothing about calibration; that takes dozens.

Outsiders inside OpenAI (55%: OpenAI names an outside organisation with ongoing access while its models train). Anthropic’s METR arrangement (story 3) is exactly this kind of agreement, but the call needs OpenAI. OpenAI said on 9 October it would announce details “in the coming weeks” and has named no partner since. Still open.

FTC filing (20%: a public FTC or federal court filing naming OpenAI or Anthropic, from the FTC’s investigation, by 30 June 2027). The subpoenas in story 2 are a step in that direction, but a subpoena is not a public filing, so the call stays open.

Memory deposits (65%: another US-listed memory or storage maker reports at least $5 billion of customer advances and refundable deposits). Western Digital reports on Thursday. Still open.

The other thirteen are unchanged. Every call’s full wording and status is at durabilitycurve.com/calls.

Calls so far: 19 open · 1 resolved, 1 true · too few to judge calibration

For each AI agent you run, what do you check besides its own account of what it did? Reply and tell me.

Free kit: Before You Trust Your AI Agent

The newsletter · Free

Subscribe to get the next structural lens in your inbox.

Several essays a week, plus Notes through the week.

$ Email address Subscribe

Signup completes on Substack. Unsubscribe from any email.

Share this essay

XLinkedInBlueskyEmail

Comments are on the Substack copy. If an essay was worth one, buy me a coffee.

Claude and ChatGPT can search this archive and its claim ledgers once you add the connector.