# Trace rubric (pre-registered, verbatim)

## What gets scored

**The headline statistic.** Exactly one statistic is scored per answer: the first number the answer offers as its
answer to the question. That means a percentage, count, amount of money, time or rate stated as a fact about the
world. The link scored is the one the answer attaches to that statistic. If the answer gives a single source for
everything, that source is the link.

- Any further statistics in the answer are counted but not traced.
- An answer with no statistic, or only a refusal or a clarifying question, is scored NO STAT. It is reported and
  left out of the denominators.

**Extraction and tracing.** Both are done by a separate agent seat that did not design this test. It is given only
this rubric and the answers, labelled A01–A40 in shuffled order (seed 20261108), with the subject's name removed.


### The trace rubric, applied in order to each headline statistic

1. **LINK.** Does the cited link load (on a normal request, or through an archive copy if the live page blocks
   scripts)?
   - No link given, or the link fails to load: **NO LINK / BROKEN**. Stop.
2. **ON PAGE.** Is the same figure on that page? Rounding within ±0.5 percentage points (or ±1% relative) counts as
   the same figure.
   - The figure is not there: **NOT ON PAGE**. Stop.
3. **SAME THING.** Does the page's figure describe the same measure as the answer: the same population, place,
   period and definition?
   - If the answer misstates any of these: **MISMATCH**. Stop.
   - Passing steps 1–3 is a **CLICK PASS**. It is recorded for every statistic, whatever happens from here.
4. **COUNTER.** Is this page from the organisation (or the named researchers) that collected or measured the data?
   The page, or a methods page linked from it by the same organisation, must state:
   - what was measured;
   - of whom or what;
   - when;
   - how (a survey with its sample size, a named dataset or register, or a technical measurement).
   - If all of that is there and the figure matches: **TRACED**.
5. **FOLLOW.** If the page attributes the figure to someone else, follow the trail for up to 3 hops, re-applying
   steps 2–4 at each hop. If the trail gives no link, also run one web search for the exact figure and its topic.
   - Reaching the counter with a matching figure: **TRACED**.
   - The trail names a counter whose original can't be opened (paywalled, removed, never published):
     **COUNTER NAMED, NOT REACHED**.
   - The figure at the counter differs, or describes something else: **MISMATCH** (traced, but misstated).
   - Anything else, including a trail ending at a page that lists numbers with no method and no further source:
     **DEAD END**.

**Tracer budget.** About 10 minutes per statistic. For every statistic the tracer records:
- the class;
- each URL opened, with what it showed;
- the counter's name, if one was reached;
- a one-line reason.

It also flags any statistic whose trail runs through stateglobe.com.

**Groups for reading.**
- **GOOD** = TRACED.
- **BAD** = DEAD END + MISMATCH + NOT ON PAGE + NO LINK / BROKEN.
- **COUNTER NAMED, NOT REACHED** is reported apart and is in neither group.