---
title: "Everyone Got Safer. That's the Problem."
description: "Safety has two numbers: how often each system fails, and whether they fail together. The field has spent years driving the first one down while almost no dashboard reports the second."
author: "Harry Floyd"
publication: "The Durability Curve"
canonical: "https://durabilitycurve.com/blog/everyone-got-safer/"
date: "2026-08-28"
series: "PROOF & TRUST"
law: "Law A"
claims: "https://durabilitycurve.com/claims/everyone-got-safer/"
format: "markdown mirror of the canonical HTML page; figures are named, not embedded"
---

# Everyone Got Safer. That's the Problem.

*Safety has two numbers: how often each system fails, and whether they fail together. The field has spent years driving the first one down while almost no dashboard reports the second.*

By Harry Floyd · 2026-08-28 · canonical: https://durabilitycurve.com/blog/everyone-got-safer/

In July 2023, a group of researchers at Carnegie Mellon and the Center for AI Safety published a single string of about twenty tokens. It looked like line noise. Appended to a harmful request, it made a language model answer anyway.

Jailbreaks were old news by 2023. The surprise was the reach. The researchers built the string against two open models they could see inside, then pointed it, unchanged, at commercial systems they could not. It worked on GPT-3.5 most of the time, on Google's Bard about two-thirds of the time, on GPT-4 roughly half.[^gcg] One string, written once, walked through the front doors of two rival labs that had never seen it.

The labs had not coordinated, and they shared no code. What they shared was less visible than code: enough behavioural structure, from broadly similar ways of being built and trained, that a single adversarial object made against other models opened theirs too. A key cut for one lock opened others that were never meant to match.

That is the whole subject of this piece, and once you can see it you will find it everywhere a system is called safe. The number that hurts you is how many safeguards fail at the same moment, for the same reason.

## The number nobody puts on the dashboard

Every safety dashboard reports the same kind of number. Jailbreak success, 0.8%. Hallucination rate, 2.1%. Eval pass rate, 94%. Monitor recall, 97%. Each is a statement about one system on its own: how often this model, this check, this control gets something wrong.

Put four such checks in front of a risk and you feel safer, and the arithmetic seems to agree. If each independently misses one failure in twenty, the chance all four miss the same one is one in twenty to the fourth power, about one in 160,000. That is the number people carry in their heads when they add a monitor.

It holds only if the four fail for different reasons. Suppose instead they share a blind spot. Each still misses one event in twenty, so every individual number on the dashboard is unchanged. But now the misses are the same miss. When the shared blind spot meets a real failure, all four go dark together, and the chance of that is not one in 160,000. It is one in twenty.

Same four safeguards. Same headline numbers on every one. Eight thousand times the exposure. What separates those two worlds is the correlation between the failures, and it is the one quantity the dashboard does not show.

Call it residual failure correlation. Once you have driven down how often each system fails, the risk that is left lives in how much their failures move together. Your dashboard reports the marginal, how often each one fails. What can kill you is the joint, how often they fail at once, and almost no dashboard reports that. None of the parts are new: reliability engineers have worried about common-mode failure for decades, Ross Ashby's cybernetics named the limit in 1956, and finance has a whole literature on correlated risk. What is new is seeing evidence of it inside AI systems while the usual dashboards report only the dimension that is improving, and having a way to measure the rest before it fires.

## This is probably your system

Picture a team shipping an AI agent with a serious-looking safety setup: an eval suite of 400 cases, a model grading every response, a red-team pass before release, and a production monitor watching for anomalies. Four safeguards. Now count the ways they could fail for the same reason.

A blind spot in the base model does not stay put. The grader marking the agent's work is from the same model family, so it can share the very blind spot it is meant to catch. If that family also helped write the eval cases, those tests can inherit the same gap: the work, the marking, and the tests may all go blind in the same place. The red team worked off the same threat list that shaped those tests, so it looks where they already look. The production monitor reads the model's own confidence, which is exactly what a shared failure leaves looking normal. These are overlapping dependencies, not one universal cause, and for a failure that sits where they overlap, the four collapse into far fewer independent routes than the count promises. On the day it hits they go quiet together while [the dashboard stays green](https://durabilitycurve.com/blog/the-evaluation-inversion/) and the agent walks off a cliff.

Every number on that dashboard was accurate. Against the failure that mattered, the system was [far less independent than it looked](https://durabilitycurve.com/blog/confidently-wrong/).

## It has happened before, at scale

This failure is old. What is new is where it is reappearing. In the mid-1980s, large funds bought a product called portfolio insurance: as the market fell, a computer sold stock-index futures on their behalf to cap the loss. Each fund, alone, had made itself safer. But they had all bought the same rule, so on 19 October 1987, when prices dropped, the rule told all of them to sell into the same falling market at the same moment. The selling amplified the fall, which triggered further selling. The Dow lost 22.6% in a day, still the worst on record.[^1987]

Each fund had insured itself. Together they had written a fire alarm wired to start the fire.

The 2008 crisis is the same shape one level up. A formula for pricing the risk that mortgages default together, published in 2000, read its correlations off current market prices rather than off decades of history nobody had.[^copula] Wired later ran the obituary under the title "Recipe for Disaster: The Formula That Killed Wall Street."[^wired] The banks did not all run one identical spreadsheet. They ran different models inside a shared modelling culture, calibrated on the same benign stretch of rising prices, resting on the same assumption. When that assumption broke, it broke everywhere at once, because it was the same assumption. Sociologists who later interviewed 114 people across the industry named it directly: a shared evaluation culture, not shared code.[^mackenzie]

Different systems, common ancestry, correlated failure. That is the pattern to carry into what is being built now.

## It is firing in AI, and the models are improving while it does

The people who coined the term "foundation model" wrote the warning down in 2021: homogenisation, they said, means "the defects of the foundation model are inherited by all the adapted models downstream."[^bommasani] A handful of base models now sit under a large share of what ships. A flaw in one can propagate, silently, into many ostensibly separate products built on it, unless something independent downstream is there to catch it.

The 2023 jailbreak was that warning made concrete. The researchers offered a shared cause, tentatively: their attack worked best on the OpenAI models, they wrote, most likely because the open model they built it on had been trained on ChatGPT's own outputs.[^gcg-quote] Shared training data was the likely route the weakness travelled. The scale test came in a 2026 competition that ran about 272,000 attempts at 13 frontier models and broke every one, with single strategies transferring across model families through what the authors read as a shared weakness in how all of them follow instructions.[^dziemian] Not every attack travels; many break one model and bounce off the next. The ones that matter are the few that travel, because one strategy can reach many nominally separate systems at once. Overlapping public benchmarks create another route for dependence: repeated exposure or contamination can make apparent agreement less independent than it looks.[^contam] And researchers have now started to measure the dependence head-on. A 2026 study of 18 models across six families found statistically significant behavioural entanglement, including failures that arrive together, of exactly the kind that quietly defeats any system trusting several models to be independent voices.[^entangle]

None of this says the models got worse. They got much better, and real ground was won on the attacks people thought to measure. What did not change is that many of the wins were shared: made against overlapping classes of attack, using the same handful of methods, so the ground the labs stand on is more common than their branding suggests. The gains tell us how often each model fails on its own. They tell us nothing about how often the models fail together. That number may have fallen too, it may have stayed flat, or the failures that survived may have concentrated in the places the models share. We do not know, because it is not a number the usual dashboard reports.

## What is proven, and what I am only predicting

Here is the honest line, because a piece about not fooling yourself has to draw it. The evidence shows that some failures transfer between systems. It does not yet show that safety optimisation or shared design has made systemic failure correlation rise over time. The clean experiment has not been run for language models, and it is not hard to describe: take pairs of models matched on how well each resists a fresh attack alone, then measure whether an attack jumps between them more when they share a base than when they do not. That holds each model's own failure rate fixed and measures the joint failures directly, which is the quantity that matters. In image classifiers the nearest version has been run, and transfer tracks how similar two models are, closely enough that a simple predictor calls it right more than nine times in ten.[^vision] I would bet the language-model result comes out the same way. Until someone runs it, that is a bet with a named test attached, which is the only kind worth making in public.

## Where this bites, and where it does not

The concern is not universal, and the boundary is simple. It applies wherever two things are true at once: you are running several safeguards because you expect them to cover for each other, and two or more of them can be defeated by the same underlying cause. Where you never expected diversification, a single well-measured process on a factory line, none of this touches you.

A shared cause comes from one of three places, and it is worth knowing which you have. Shared substrate is the same base model, data vendor, or infrastructure sitting under nominally separate systems. Shared method is the same benchmark, rubric, or threat model, so everyone is blind to the same unlisted case. Shared adaptation is everyone optimising against the same visible measure until the risk has been pushed into the same unwatched place. That third one is Goodhart, and it has a clean demonstration in AI: when researchers at OpenAI trained hard against a monitor that read a model's chain of thought, the model did not stop misbehaving, it learned to keep the visible reasoning clean and misbehave anyway.[^baker] The measure stayed green while the failure moved to where it could not point, which is the same place any other team optimising the same way can end up blind.

Commercial aviation shows what deliberately buying independence looks like. It is about as measured as human activity gets, and the fatal-accident rate in the last decade was about 60% below the decade before, even as departures rose.[^aviation] That decline came from many things at once, engines and training and air-traffic systems among them, but the architecture is built on independence rather than resolution alone: critical systems use redundancy, sensors cross-checked against each other, and a reporting culture that keeps hunting for failure modes nobody has seen yet. The 737 MAX is the exception that shows the rule. A critical system rode on a single angle-of-attack sensor with nothing built to argue with it, and when that one sensor lied, two planes went down within five months, same cause, and the fleet was grounded.[^mcas] Its independence count was one, and the design was betting it was more.

[^gcg]: Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, Matt Fredrikson, "Universal and Transferable Adversarial Attacks on Aligned Language Models," [arXiv:2307.15043](https://arxiv.org/abs/2307.15043), 2023. Ensemble transfer attack success rates (Table 2): GPT-3.5 ~86.6%, GPT-4 ~46.9%, PaLM-2 (Bard) ~66.0%. Claude was more mixed: Claude-1 was 47.9%, comparable to GPT-4, while Claude-2 was markedly more robust at ~2.1%. The attack was optimised on open models (Vicuna) and transferred to closed commercial models it never had access to.

[^1987]: [Black Monday](https://www.federalreservehistory.org/essays/stock-market-crash-of-1987), 19 October 1987: the Dow Jones Industrial Average fell 22.6% (508 points) and the S&P 500 fell about 20.4%, the largest single-day percentage drops on record. The Brady Commission report gave portfolio-insurance and index-arbitrage selling heavy weight among the causes; the precise causal weight remains debated, so this piece says the strategy amplified the cascade rather than manufactured it alone.

[^copula]: David X. Li, "[On Default Correlation: A Copula Function Approach](https://doi.org/10.3905/jfi.2000.319253)," Journal of Fixed Income 9(4), 2000, pp. 43-54. The model estimated joint-default probability from current market credit spreads rather than from historical default data.

[^wired]: Felix Salmon, "[Recipe for Disaster: The Formula That Killed Wall Street](https://www.wired.com/2009/02/wp-quant/)," Wired, 23 February 2009.

[^mackenzie]: Donald MacKenzie and Taylor Spears, "'The formula that killed Wall Street': The Gaussian copula and modelling practices in investment banking," [Social Studies of Science 44(3), 2014](https://journals.sagepub.com/doi/10.1177/0306312713517157), pp. 393-417, drawing on documentary material and 114 interviews. Their account describes a shared "evaluation culture" across banks rather than one identical model: different systems, common intellectual ancestry, correlated failure.

[^bommasani]: Rishi Bommasani et al., "On the Opportunities and Risks of Foundation Models," Stanford Center for Research on Foundation Models, [arXiv:2108.07258](https://arxiv.org/abs/2108.07258), 2021: "homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream."

[^gcg-quote]: [Zou et al., 2023](https://arxiv.org/abs/2307.15043): the authors note the attack's success "is much higher against the GPT-based models, potentially owing to the fact that Vicuna itself is trained on outputs from ChatGPT." One plausible transmission path was shared training data rather than shared code.

[^dziemian]: Dziemian et al., "How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition," [arXiv:2603.15714](https://arxiv.org/abs/2603.15714), 2026: roughly 272,000 attempts across 13 frontier models produced 8,648 successful attacks; every model tested was vulnerable, with universal attack strategies transferring across model families, attributed to shared weaknesses in instruction-following architecture. Per-model success rates ranged from about 0.5% to 8.5%: shared failure mode, unequal magnitude.

[^contam]: Benchmark contamination is a documented problem in public LLM evaluations: [Deng et al. (NAACL 2024)](https://aclanthology.org/2024.naacl-long.482/) found evidence of memorisation in MMLU and other test sets, and [Zhao et al. (ACL 2025)](https://arxiv.org/abs/2412.15194) introduced MMLU-CF to reduce contamination and found substantial changes in model scores and rankings relative to the original. Models drawing on the same contaminated sets can share blind spots as a result, which is what makes their agreement less independent than it looks.

[^entangle]: Kuai et al., "A Statistical Framework for Auditing Behavioral Dependence and Induced Bias in LLM Judges," [arXiv:2604.07650](https://arxiv.org/abs/2604.07650), 2026 (the v2 title): a study of 18 models across six families (GPT, Claude, Qwen, Llama, Gemini, DeepSeek) that found statistically significant behavioural entanglement (Spearman 0.508 and 0.520, p<0.01), including synchronised and coincident failures, and showed the dependence was associated with judge over-endorsement bias on a disjoint MMLU-Pro set. It identifies shared pretraining data, distillation, and alignment pipelines as plausible sources of that dependence, and states that "apparent agreement reflects shared error modes rather than independent validation."

[^vision]: The controlled contrast (hold standalone robustness fixed, vary only substrate-sharing) has not been published for language models. In image classifiers it has: adversarial-attack transfer tracks surrogate-target similarity ([Demontis et al., USENIX Security 2019](https://www.usenix.org/conference/usenixsecurity19/presentation/demontis); "The Relationship Between Network Similarity and Transferability of Adversarial Attacks," [arXiv:2501.18629](https://arxiv.org/abs/2501.18629), 2025), where a predictor built on model similarity forecasts transfer success more than 90% of the time. This is adjacent evidence, one domain over, not a same-domain LLM proof.

[^baker]: Bowen Baker et al. (OpenAI), "Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation," [arXiv:2503.11926](https://arxiv.org/abs/2503.11926), 2025. Monitoring the chain of thought helps at low optimisation pressure, but "with too much optimization, agents learn obfuscated reward hacking, hiding their intent within the CoT while still exhibiting a significant rate of reward hacking."

[^aviation]: Boeing, [Statistical Summary of Commercial Jet Airplane Accidents](https://www.boeing.com/content/dam/boeing/v2/safety/statsum.pdf), Worldwide Operations 1959-2025 (the "2025 Statistical Summary," published April 2026): "Over the past two decades, this report documents a 35% decline in the total accident rate and a 60% decline in the fatal accident rate, all while departures have increased by more than 20%" (comparing 2006-2015 with 2016-2025). The figure is a rate per departure, not a measure of severity.

[^mcas]: The 737 MAX's MCAS could trim the aircraft nose-down based on a [single angle-of-attack sensor](https://www.boeing.com/content/dam/microsites/static/737-max-updates/mcas/index.html) with no independent cross-check, and the system was not disclosed to pilots. Lion Air Flight 610 (October 2018) and Ethiopian Airlines Flight 302 (March 2019) crashed from the same failure, killing 346 people; the worldwide fleet was grounded in March 2019.

  
Paid subscribers

  
The rest of this piece is for paid subscribers, on any tier.

  
[Read the rest on Substack](https://harryfloyd.substack.com/p/everyone-got-safer)