# PREREG — survives-a-recheck — SCORE cue test — 2026-10-27

**Why.** Review wave 1's red team found that the piece ignores SCORE. SCORE (Tyner et al., "Investigating the
replicability of the social and behavioural sciences", *Nature*, 1 April 2026) re-ran 274 claims from 164 papers in
54 journals (2009–2018), sampled from a stratified random draw of papers, at a median 99.6% power. That is the "next
large batch of re-runs" the piece's two published falsifiers name:
- **The p-value cue:** "If the next large batch of re-runs shows first results below .001 failing as often as those
  near .05, this cue is wrong."
- **Sample size:** "If a pre-registered test on a new pool finds the biggest original samples holding up at least 10
  points more often than the smallest, the point about sample size is wrong."

The claim-level data are public under CC0 (OSF dtzx4), so the falsifiers are tested here, not deferred.

## What is known before writing

**SCORE's published figures.** Verified by a research pass:
- 151 of 274 claims replicated (55.1%).
- The original p-value correlates −.11 [−.26, .04] with replication at paper level, and −.18 [−.29, −.05] at claim
  level (MetaArXiv 7u58q, Table 3 and SI Table S19).
- No results by p-value band are published.

**Inspected by the author.** Only predictor files, no outcomes:
- `orig_outcomes.csv`: 825 original claims with p-value type and value, test statistic and sample size. It holds no
  replication result.
- `paper_metadata.csv`: discipline category.

From those files, p-value types are 405 thresholds and 319 exact values, and the test statistics are 264 t, 94 z, 73 F
and 15 χ². The FReD rule places the 825 claims as S 244, W 105, M 75, UNRES 243, MISSING 97 and NS 61. The replicated
subset is not known.

**Not opened:** `repli_outcomes.csv` and `repli_exclusions.csv`. They are downloaded only after this file is locked,
and their sha256 is recorded in RESULTS.

## Data

**Originals.** OSF dtzx4, `/Analysis-ready Data/Replication and Reproduction Papers/`, copies in
`data/score/`:
- `orig_outcomes.csv`, sha256 `325b445d…f26478f`
- `paper_metadata.csv`, sha256 `0f2beef4…a5934c660e`

Both hashes are pinned in the script.

**Replications.** `repli_outcomes.csv` (https://osf.io/download/2c8gh/) and `repli_exclusions.csv`
(https://osf.io/download/wq65j/):
- One row per claim: the version of record (`repli_version_of_record`); otherwise the first report by `report_id`.
- Listed exclusions are removed.
- **Outcome:** `repli_score_criteria_met`, SCORE's own criterion (significant at p < .05, two-tailed, in the original
  pattern). Unparseable values are excluded and counted.
- **Expected:** about 274 claims, which is SCORE's published count. A difference is reported, not corrected.

## Tests (all reuse `fred_two_cues.py`'s band rule, paper-clustered bootstrap and verdict thresholds unchanged)

**Shared settings:**
- **Weighting (primary):** each paper weighted once, as in the FReD test. Claim-weighted results are reported
  alongside.
- **Bootstrap:** 5,000 draws, seed 20261027.
- **Verdicts:** CUE HOLDS means a gap of at least 10 points with the 95% interval above zero. CUE REVERSED means the
  interval is below zero. Otherwise, CUE NOT SHOWN.

**A. The falsifier as published (primary for the p-value cue).**
- Band S (p < .001) against band W (.01–.05), by the FReD rule (`p_band`). A bare threshold such as "< .05" or "< .01"
  is unresolved and left out, as in the FReD test.
- **The falsifier FIRES** if the point estimate of S − W is zero or below: strong passes failed as often as narrow
  ones.
- If the gap is positive but the verdict is CUE NOT SHOWN, it does not fire. The piece must then say SCORE points the
  same way without showing it.

**B. The card's first sign as a reader applies it (co-primary for the card).**
- **Strong:**
  - an exact printed p below .001; or
  - a threshold at or below .001; or
  - no exact p, and the test statistic gives p < .001 two-tailed. The usable statistics are t with df, z, F with one
    numerator df (t = √F) and χ² with one df. A t with no df must exceed 3.65.
- **Weak:** everything else significant. With no usable statistic, the sign counts, as the card says.
- Strong against weak, same verdict rule.

**C. Sample size (primary for the second falsifier).**
- The biggest third of original samples against the smallest third, among significant originals (`run_h2`).
- **The falsifier FIRES** if the verdict is SIZE HELPS: at least 10 points, with the interval above zero.

**Secondary (reported, no verdict binds):**
- the psychology-and-health category alone;
- the claim-weighted A;
- the middle band's (.001–.01) rate.

## What the piece does with each result (committed now)

- **A: CUE HOLDS.** §4 reports SCORE beside FReD as the large batch the falsifier named, with the cue holding. The
  falsifier sentence says it has now been tested once.
- **A: CUE NOT SHOWN, gap above zero.**
  - §4 reports SCORE plainly: same direction, not clearly shown, and the interval stated.
  - The pull quote and the card keep the cue, but the piece calls it "shown in one large pool, pointing the same way
    in a second".
- **A: FIRES (gap at or below zero).**
  - The piece says plainly that the cue failed its own falsifier on SCORE.
  - The card's first sign is presented as unconfirmed.
  - The piece goes back to the operator before Stage 6, to re-spec or park. It is not quietly revised.
- **B** is reported in §5 beside the card. If B is NOT SHOWN or REVERSED while A holds, the piece says the reader's
  version of the sign was weaker than the band test.
- **C: FIRES.** The piece withdraws "bigger samples did not replicate detectably better" for SCORE and says so. If it
  does not fire, the sample-size point stands, with SCORE's figure added.
- **Every result:** the falsifier checks are recorded for the track record, whichever way they land.

## Controls (run before this file; the replication file was never read)

- **`--dummy` (random outcomes on all 825 originals):** A +2 [−12, +16] NOT SHOWN; B −5 NOT SHOWN; C +3, no advantage.
- **`--plant` (S at 65%, the rest at 45%):** A HOLDS (all fields and psychology). Claim-weighted +16 [+4, +28].
- **Outputs:** `score-cue-test-results.dummy.json` and `.plant.json`.

## Scoring

- **Script:** `score_cue_test.py`, sha256
  `07a1b78b8566314ba8b4411af60c6eaed18f478bc19f3accaf665f6737228897`. It imports `fred_two_cues.py` (sha256
  `c86f14ba4b9be3a13994d5c2c86581e173e771b5ff31e8280953cbff820ddd19`, unchanged since its own lock).
- **Order:** lock → `verify` → download the two replication files and record their sha256 → hash check on the
  scripts → real run.
- **Results:** appended to RESULTS — survives-a-recheck — FReD two cues — 2026-10-26.