Principle A · built from Law II, Law IV and Law V
Goodhart Corollary
Push hard on a number that only stands in for what you want, and past a point the number keeps rising while the real thing falls. The two usually track at first, then part company, and a better stand-in does less damage. The exception is a number that fully captures the thing, which you usually cannot know in advance.
- StandingRevised under pressure, relied on
- Last tested2 Sept 2026
- Revised3 times
- AppliesAt any time scale
Narrowed after a test in September 2026. The narrowed wording is the live test.
Why it is its own principleThe laws it is built from explain the miss. What this adds is a loop: the optimiser does not just miss the target, it wears down the very signal it is chasing.
Ask this of your own work
If this number keeps going up, at what point would it stop meaning what you want, and what separate check would tell you?
The rule it gives youOnce the thing being measured can tell the test from the task, a steady score is no evidence of steady behaviour. Check how it behaves in use, by a route the score does not touch.
An example
Test scores rise when teaching narrows to the test, and hospital report cards changed which patients surgeons would take on. By contrast, better ImageNet accuracy largely carried over to other image tasks, a stand-in close to complete; even that exception is contested.
Run it on your own work
Start with
- The Stable Liar Every metric you optimise quietly stops measuring what you meant. The dangerous ones never break. They keep reporting green while the thing underneath rots.
- Ten Lines of Code Scored 100%. One Agent Broke Eight Benchmarks. Not one task was actually solved, and the same blind spot is sitting in your own dashboard.
- You Cannot Try to Fall Asleep Sleep is only where you notice it first. Much of what matters works the same way.
The other essay under Principle A
How it could be wrong
What would break it
What would count as proof it is wrong. When a test meets the old version, the law is rewritten and the change is recorded beside it.
For ordinary optimisers
Sustained, hard optimisation of a stand-in that can be gamed and is not complete, which still improved the real thing, or at least did not degrade it, and never parted from it.
For systems that can tell the test from the task
A capable system with a reason to behave well only when watched, which kept passing its evaluation and kept behaving well in use, checked by something it could not model. Or a more capable system that became easier to evaluate under the same test.
The first was rewritten on 2 Sept 2026, when a test met the old version.
What changed, and why
· extended, not after a test
Widened to capable systems: once the thing being measured can tell the test apart from the task, the score can look healthy while behaviour in use drifts, and improving the test does not help, because the system can model the improvement too.
· extended, not after a test
Checks have to come from different layers (the contract, the thing measured, the trace of what happened), because checks within one layer share the same weak spot.
· after a test
A test of five cases found the old wording wrong twice. It said the damage grows steadily with the optimiser’s capability; in fact the real thing usually tracks the number at first and parts company later, and a better stand-in does less damage. And “any number degrades” was too strong: ImageNet accuracy improved what it stood for, a contested exception. The core held.
The exact wording, as the framework records it
Optimising an imperfect proxy degrades the construct it stands for — non-monotonically — unless the proxy is a sufficient statistic for it (which you usually cannot know in advance).
The full statement
Any metric used as an optimization target degrades the construct it stands for under sustained optimization pressure — UNLESS the proxy is a sufficient statistic for the construct, in which case optimizing the proxy improves it (a condition rarely total and usually not knowable in advance). AMENDED 2026-09-02: the degradation is NON-MONOTONIC — the construct typically tracks the proxy early and decouples only past an over-optimization point (an inverted-U), and its severity falls as proxy quality rises; the prior "erodes monotonically with optimizer capability" is WITHDRAWN (the trajectory is non-monotone, and the coefficient that actually scales is proxy quality, which reduces corruption). The gaming strategies generalise and compose.