{
 "ledger": {
  "title": "Claim Ledger: Before You Trust a Study, Check How Strongly It Passed",
  "url": "https://durabilitycurve.com/claims/survives-a-recheck/",
  "essay": {
   "title": "Before You Trust a Study, Check How Strongly It Passed",
   "url": "https://durabilitycurve.com/blog/survives-a-recheck/",
   "published": "2026-11-01"
  },
  "last_checked": "2026-10-27",
  "format": "current",
  "counts": {
   "claims": 67,
   "by_status": {
    "VERIFIED": 49,
    "REPORTED": 3,
    "EXECUTED": 15
   },
   "verified_at_primary": 49,
   "removed": 0
  },
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "attribution": "The Durability Curve (Harry Floyd), https://durabilitycurve.com/claims/survives-a-recheck/"
 },
 "rows": [
  {
   "id": "A01b",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A01b",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Replication effects were about half the original: mean r 0.197 vs 0.403",
   "quote": "\"Replication effects were half the magnitude of original effects … original study effect sizes ( M = 0.403, SD = 0.188) were reliably larger than replication effect sizes ( M = 0.197, SD = 0.257)\"",
   "removal_reason": "",
   "source": "OSC 2015, accepted manuscript (OSF)",
   "source_url": "https://osf.io/447b3",
   "locator": "Abstract (PDF p. 1); Results, effect sizes (PDF p. 18)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A01c",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A01c",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Originals with p < .001: 20 of 32 replicated",
   "quote": "\"Almost two thirds (20 of 32, 63%) of original studies with P < … 0.001 had a significant P value in the replication.\"",
   "removal_reason": "",
   "source": "OSC 2015, accepted manuscript (OSF)",
   "source_url": "https://osf.io/447b3",
   "locator": "Results, correlates of reproducibility (PDF pp. 20–21; the sentence crosses a page break)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A01d",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A01d",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Originals with p > .04: 2 of 11 replicated",
   "quote": "\"whereas 6 of 23 (26%) that had a P value between 0.02 < P < 0.04 and 2 of 11 (18%) that had a P value > 0.04 did so\"",
   "removal_reason": "",
   "source": "OSC 2015, accepted manuscript (OSF)",
   "source_url": "https://osf.io/447b3",
   "locator": "Results (PDF p. 20)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A01e",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A01e",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Main or simple effects replicated 23/49; interactions 8/37",
   "quote": "\"Among original, significant effects, 23 of the 49 (47%) that tested main or simple effects replicated at P < 0.05, but just 8 of the 37 (22%) that tested interaction effects did.\"",
   "removal_reason": "",
   "source": "OSC 2015, accepted manuscript (OSF)",
   "source_url": "https://osf.io/447b3",
   "locator": "Results (PDF p. 20)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A01f",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A01f",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Rated surprisingness correlated −0.244 (Spearman) with replication significance",
   "quote": "\"Surprising original result -0.244 0.102 -0.181 -0.113 -0.241 … Surprising effects were less reproducible\"",
   "removal_reason": "",
   "source": "OSC 2015, accepted manuscript (OSF)",
   "source_url": "https://osf.io/447b3",
   "locator": "Table 2, first column = \"Replications P < .05 in original direction\" (PDF p. 12); Results (PDF p. 22)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A04a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A04a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Ego-depletion RRR: 23 labs, N = 2,141, d = 0.04 [−0.07, 0.15]",
   "quote": "\"Multiple laboratories (k = 23, total N = 2,141) conducted replications of a standardized ego-depletion protocol … (d = 0.04, 95% CI [-0.07, 0.15]\"",
   "removal_reason": "",
   "source": "Hagger et al. (2016), Perspectives on Psychological Science 11(4):546–573. Published SAGE PDF from the Univ. Konstanz repository (KOPS)",
   "source_url": "https://kops.uni-konstanz.de/server/api/core/bitstreams/d1f20ce9-761f-40e6-aaa8-4992d4744d3c/content",
   "locator": "Abstract, p. 546",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A07a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A07a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "27 findings judged by 233 people without a PhD in psychology: 59% correct",
   "quote": "\"Each of 27 high-profile social-science findings was evaluated by 233 people without a Ph.D. in psychology. Results showed that these laypeople predicted replication success with above-chance accuracy (i.e., 59%).\"",
   "removal_reason": "",
   "source": "Hoogeveen, Sarafoglou & Wagenmakers (2020), AMPPS 3(3):267–285. Final published version (UvA repository)",
   "source_url": "https://pure.uva.nl/ws/files/53717212/2515245920919667.pdf",
   "locator": "Abstract, p. 267",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A07b",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A07b",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "67% correct when shown the strength of evidence",
   "quote": "\"when participants were informed about the strength of evidence from the original studies, this boosted their prediction performance to 67%.\"",
   "removal_reason": "",
   "source": "Hoogeveen et al. (2020), published version",
   "source_url": "https://pure.uva.nl/ws/files/53717212/2515245920919667.pdf",
   "locator": "Abstract, p. 267; group-level ω .59 [.57, .60] vs .67 [.65, .69] on p. 275",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A07c",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A07c",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Two separate groups: 123 people in the description-only condition and 110 in the description-plus-evidence condition; the two accuracy figures in A07a and A07b come from these two groups",
   "quote": "\"the final sample consisted of 233 participants, 123 in the description-only condition and 110 in the\"",
   "removal_reason": "",
   "source": "Hoogeveen et al. (2020), published version",
   "source_url": "https://pure.uva.nl/ws/files/53717212/2515245920919667.pdf",
   "locator": "Method, Participants, p. 269",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A07d",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A07d",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Evidence strength shown as a Bayes factor plus a verbal label",
   "quote": "\"the Bayes factor and its verbal interpretation (e.g., \"moderate evidence\") were added to the description of each study.\"",
   "removal_reason": "",
   "source": "Hoogeveen et al. (2020), published version",
   "source_url": "https://pure.uva.nl/ws/files/53717212/2515245920919667.pdf",
   "locator": "Method, Materials, p. 270; labels follow Jeffreys (1939) as adjusted by Lee & Wagenmakers (2013)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A09c",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A09c",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "215 surveyed psychologists ranked \"large sample size\" first (M = 3.84)",
   "quote": "\"Respondents rated \"large sample size\" as the most important criterion (M = 3.84), judging it to be significantly more important than the second most important factor, \"large effect size\" (M = 3.61)\" [...] \"resulting in a final sample size of 215 psychological scientists\"",
   "removal_reason": "",
   "source": "Li et al. (2024), publisher PDF",
   "source_url": "https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0306911&type=printable",
   "locator": "Survey results, p. 10 of 15 (n = 215 on the same page)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A12c",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A12c",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "At 33% power (t tests with 38 degrees of freedom), 57.6% of significant p values are greater than .01",
   "quote": "\"t tests with 38 degrees of freedom and powered at 33% will result in 57.6% of significant p values being greater than\" [decoded: the text layer reads greater than .0l; the page reads .01]",
   "removal_reason": "",
   "source": "Simonsohn et al. (2014), published PDF",
   "source_url": "https://gwern.net/doc/statistics/meta-analysis/2014-simonsohn.pdf",
   "locator": "p. 538 (the text continues \".01\"; the same passage puts 10.4% above .04); note (derived): so 42.4% fall below .01",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A14a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A14a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Many Smiles: n = 3,878 across 19 countries; mimicry and voluntary posing both amplified and initiated happiness",
   "quote": "\"Data from n = 3,878 participants spanning 19 countries indicated that a facial mimicry and voluntary facial action task could both amplify and initiate feelings of happiness.\"",
   "removal_reason": "",
   "source": "Coles et al. (2022), Nature Human Behaviour 6:1731–1742. Published abstract on the publisher page (full text paywalled)",
   "source_url": "https://www.nature.com/articles/s41562-022-01458-9",
   "locator": "Abstract",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "A14b",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A14b",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Pen-in-mouth evidence was less conclusive",
   "quote": "\"However, evidence of facial feedback effects was less conclusive when facial feedback was manipulated unobtrusively via a pen-in-mouth task.\"",
   "removal_reason": "",
   "source": "Coles et al. (2022), published abstract",
   "source_url": "https://www.nature.com/articles/s41562-022-01458-9",
   "locator": "Abstract",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B1o",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B1o",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Ego depletion. Baumeister, Bratslavsky, Muraven & Tice (1998), Experiment 1: hungry students ate radishes while resisting chocolates (or ate chocolates, or no-food control), then persistence on unsolvable puzzles was timed. N = 67. Key test as reported: one-way ANOVA across the three conditions, F(2, 64) = 26.88, p < .001; radish vs chocolate t(44) = 6.03, p < .001. Design: SIMPLE (one manipulation, one outcome: persistence time).",
   "quote": "\"Data were collected in individual sessions from 67 introductory psychology students\" [...] \"A one-way analysis of variance (ANOVA) indicated significant variation among the three conditions, F(2, 64) = 26.88, p < .001.\" [...] \"participants in the radish condition quit sooner on the frustrating task than did participants in either the chocolate condition, t(44) = 6.03, p < .001\"",
   "removal_reason": "",
   "source": "Baumeister et al. (1998), JPSP 74(5), 1252–1265. Published version (scan with text layer), course copy at the University of Washington.",
   "source_url": "https://faculty.washington.edu/jdb/345/345%20Articles/Baumeister%20et%20al.%20%281998%29.pdf",
   "locator": "Exp. 1, Method (Participants) PDF p. 3; Results (Persistence) PDF p. 4",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B2o",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B2o",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Strack, Martin & Stepper 1988, Study 1: Ninety-two undergraduates held a pen in the lips, the non-dominant hand or the teeth while rating cartoons; reported as a significant linear contrast, t(89) = 1.85, p = .03 (the text layer reads f(89)). Design: SIMPLE",
   "quote": "\"Subjects. Ninety-two male and female undergraduates of the University of Illinois participated in the experiment\" [...] \"The predicted differences were confirmed by a significant linear contrast, f(89) = 1.85, p = .03.\"",
   "removal_reason": "",
   "source": "Strack, Martin & Stepper (1988), JPSP 54(5), 768–777. Published APA version (scan with text layer), third-party copy on GitHub Pages.",
   "source_url": "https://jhrcook.github.io/wagenmaker-data-analysis/papers/Strack_1988_Inhibiting-and-facilitating-conditions-of-the-human-smile-a-nonobtrusive-test-of-the-facial-feedback-hypothesis.pdf",
   "locator": "Study 1, Method PDF p. 3; Results, journal p. 772 (PDF p. 5); note: means 4.32 / 4.77 / 5.14 from the paper's table, not quoted; two-tailed p for t(89) = 1.85 is about .068 (computed), so p = .03 matches a one-tailed test",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B5o",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B5o",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Caruso, Vohs, Baxter & Waytz 2013, Experiment 1: Thirty adults; a faint image of $100 bills behind the instructions; t(28) = 2.12, p = .043, d = 0.80. Design: SIMPLE",
   "quote": "\"Thirty adults from a university study pool\" [...] \"Participants assigned to the money condition saw a faint image of $100 bills in the background of the instruction screen\" [...] \"As predicted, participants in the money condition more strongly endorsed system justification than did participants in the control condition, t(28) /H110052.12, p /H11005.043, d /H110050.80\" [decoded: t(28) = 2.12, p = .043, d = 0.80; the text layer renders = as /H11005]",
   "removal_reason": "",
   "source": "Caruso, Vohs, Baxter & Waytz (2013), JEP: General 142(2), 301–306. Published APA version (online-first), copy linked from the Kellogg School research page.",
   "source_url": "http://static.squarespace.com/static/51e3f4ede4b053e5f0062efd/t/51f7e294e4b03d20c9b472cc/1375199892722/mere-exposure-to-money-increases-endorsement-of-free-market-systems-and-social-inequality%20.pdf",
   "locator": "Exp. 1, Method and Results, PDF p. 3",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B6o",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B6o",
   "status": "REPORTED",
   "status_label": "Reported",
   "removed": false,
   "claim": "Anchoring. Jacowitz & Kahneman (1995): a calibration group estimates quantities; anchored groups first judge whether a high or low anchor (15th or 85th percentile of calibration estimates) is above or below the true value, then estimate. N: not obtained. Test statistic and p: not obtained. Design: SIMPLE (provisional, from the abstract: one manipulation, anchor high vs low, on estimates).",
   "quote": "\"Subjects in the anchored condition first judge whether a specified number (the anchor) is higher or lower than the true value before estimating each quantity.\" [...] \"(15th and 85th percentiles in this study)\"",
   "removal_reason": "",
   "source": "Jacowitz & Kahneman (1995), PSPB 21(11), 1161–1166. Abstract only, reconstructed from the OpenAlex index record.",
   "source_url": "https://api.openalex.org/works/doi:10.1177/01461672952111004",
   "locator": "Abstract",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B7o",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B7o",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Karpicke & Blunt 2011, Experiment 1: Eighty undergraduates; retrieval practice vs concept mapping one week later: d = 1.50, F(1, 38) = 21.63; no p-value reported. Design: SIMPLE",
   "quote": "\"Eighty undergraduate students participated in Experiment 1.\" [...] \"the advantage of retrieval practice (M = 0.67) over elaborative studying with concept mapping (M = 0.45) represented about a 50% improvement in long-term retention scores [d = 1.50, F1,38 = 21.63\" [decoded: d = 1.50, F(1, 38) = 21.63; the text layer reads F1,38]",
   "removal_reason": "",
   "source": "Karpicke & Blunt (2011), Science 331, 772–775. Published Science version, copy on the author's Purdue lab site.",
   "source_url": "https://learninglab.psych.purdue.edu/downloads/2011/2011_Karpicke_Blunt_Science.pdf",
   "locator": "Exp. 1, journal p. 772–773 (PDF pp. 2–3); note: key comparison n = 40 (df 38)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B8o",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B8o",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Framing flips preferences. Tversky & Kahneman (1981), Problems 1 and 2 (\"Asian disease\"), separate groups. N = 152 (gain frame) and N = 155 (loss frame). Result as reported: 72% chose the sure option under the gain frame; 22% chose the sure option under the loss frame. No test statistic or p-value is reported. Design: SIMPLE (one manipulation, frame; one outcome, choice).",
   "quote": "\"Problem 1 [N = 152]: Imagine that the U.S. is preparing for the outbreak of an unusual Asian disease\" [...] \"If Program A is adopted, 200 people will be saved. [72 percent]\" [...] \"Problem 2 [N = 155]: If Program C is adopted 400 people will die. [22 percent]\" [...] \"The change is accompanied by a pronounced shift from risk aversion to risk taking.\"",
   "removal_reason": "",
   "source": "Tversky & Kahneman (1981), Science 211(4481), 453–458. Published version (JSTOR scan with text layer), copy on gwern.net.",
   "source_url": "https://gwern.net/doc/psychology/1981-tversky.pdf",
   "locator": "Problems 1–2, journal p. 453 (PDF p. 1)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B1r",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B1r",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Hagger, Chatzisarantis et al. (2016), Registered Replication Report of ego depletion. Qualifies: yes. Pre-registered: yes. Labs: 23. N = 2,141. Author overlap: none (Baumeister is listed as a protocol vetter, not an author). Task differs from B1o: depletion by a letter-\"e\" task (adapted from Baumeister et al. 1998, Study 4), outcome the multi-source interference task (MSIT); the radish task was judged too experimenter-dependent for many labs. Headline: d = 0.04, 95% CI [−0.07, 0.15].",
   "quote": "\"Participating labs preregistered their implementation plan on the Open Science Framework and conducted independent replications.\" [...] \"Multiple laboratories (k = 23, total N = 2,141) conducted replications of a standardized ego-depletion protocol based on a sequential-task paradigm by Sripada et al.\" [...] \"Meta-analysis of the studies revealed that the size of the ego-depletion effect was small with 95% confidence intervals (CIs) that encompassed zero (d = 0.04, 95% CI [−0.07, 0.15].\" [...] \"Protocol vetted by: Chandra Sripada, Daniel Kessler, Roy Baumeister\" [...] \"required participants to taste radishes and resist cookies (Baumeister et al., 1998, Study 1). This task would require extensive experimenter involvement in its delivery\" [...] \"The first task was a modified version of the letter “e” task (Baumeister et al., 1998, Study 4)\"",
   "removal_reason": "",
   "source": "Hagger et al. (2016), Perspectives on Psychological Science 11(4), 546–573. Published SAGE PDF in the University of Konstanz repository (KOPS).",
   "source_url": "https://kops.uni-konstanz.de/server/api/core/bitstreams/d1f20ce9-761f-40e6-aaa8-4992d4744d3c/content",
   "locator": "Title block and Abstract, journal p. 546; Method, pp. 548–550",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B2r",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B2r",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Wagenmakers et al. 2016 RRR: 17 independent direct replications of Strack Study 1 on one vetted protocol; smile minus pout rating difference 0.03, 95% CI -0.11 to 0.16; Strack supplied materials and early feedback and declined to review the final protocol; the neutral hand condition was omitted. Qualifies: yes",
   "quote": "\"This Registered Replication Report describes the results of 17 independent direct replications of Study 1 from Strack et al. (1988), all of which followed the same vetted protocol.\" [...] \"Data and registered protocols: https://osf.io/pkd65/\" [...] \"Our meta-analysis revealed a rating difference of 0.03 units with a 95% confidence interval ranging from −0.11 to 0.16.\" [...] \"Fritz Strack provided the original materials as well as valuable feedback and constructive suggestions during the early stages of protocol development by the lead lab.\" [...] \"Although he declined to review the final protocol\" [...] \"we omitted the neutral “nondominant hand condition” from the design\"",
   "removal_reason": "",
   "source": "Wagenmakers et al. (2016), Perspectives on Psychological Science 11(6), 917–928. Published SAGE PDF in the Maynooth University repository.",
   "source_url": "https://mural.maynoothuniversity.ie/id/eprint/15647/1/1745691616674458.pdf",
   "locator": "Title block and Abstract, journal p. 917; pp. 918–919; Table 1, p. 921; note (our sums of Table 1): 1,894 analysed, 2,533 tested",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B2s",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B2s",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Coles et al., Many Smiles Collaboration (2022), multi-site adversarial collaboration: published abstract and author list. Qualifies: yes. Pre-registered: yes. Countries: 19. N = 3,878. Author overlap: yes, Fritz Strack (first author of B2o) is a co-author. Headline (abstract): facial mimicry and voluntary facial action raised happiness; the pen-in-mouth evidence was \"less conclusive\". Outcome differs from B2o: self-reported happiness, not cartoon funniness.",
   "quote": "\"We thus formed a global adversarial collaboration and carried out a preregistered, multicentre study designed to specify and test the conditions that should most reliably produce facial feedback effects.\" [...] \"Data from n = 3,878 participants spanning 19 countries\" [...] \"However, evidence of facial feedback effects was less conclusive when facial feedback was manipulated unobtrusively via a pen-in-mouth task.\" [...] \"Fritz Strack\"",
   "removal_reason": "",
   "source": "Coles et al. (2022), Nature Human Behaviour 6, 1731–1742. Published article page on nature.com (abstract and author list are open; body paywalled).",
   "source_url": "https://www.nature.com/articles/s41562-022-01458-9",
   "locator": "Abstract; author list",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B5r",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B5r",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Many Labs 1, currency priming (pre-registered; 36 samples, 6,344 participants overall): original d = 0.80 [.05, 1.54]; replication weighted d = -0.02 [-0.08, 0.04], unweighted 0.01 [-0.06, 0.09]; proportion of samples significant in the original direction 0.03; t = -0.79; did not replicate. Qualifies: yes",
   "quote": "\"All materials, data, videos of the procedure, and the original preregistered design are available at the project page https://osf.io/ydpbf/.\" [...] \"In total, there were 36 samples and settings that collected data from a total of 6,344 participants\" [...] \"Currency priming 0.80 .05, 1.54 0.00 0.01 /C00.06, 0.09 /C00.02 /C00.08, 0.04 0.00 0.03 0.97 t = /C00.79 6,331 6,333 0.83\" [...] \"Two studies – flag priming and currency priming – did not replicate the original effects.\" [...] \"the US dollar was usually replaced with the relevant country’s currency\" [...] \"We thank Eugene Caruso\" [decoded Table 2 row: 0.80 .05, 1.54 0.00 0.01 -0.06, 0.09 -0.02 -0.08, 0.04 0.00 0.03 0.97 t = -0.79; the text layer renders minus as /C0]",
   "removal_reason": "",
   "source": "Klein et al. (2014), Social Psychology 45(3), 142–152. Published Hogrefe version (OpenMind licence), on the project's OSF page.",
   "source_url": "https://osf.io/ebmf8/",
   "locator": "Method p. 143; currency description p. 145; Table 2 p. 148; Results p. 149; Author note; note: original CI 95%, replication CI 99% (Table 2 columns); N = 6,333 for this effect",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B6r",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B6r",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Many Labs 1, anchoring (between-subjects, four of the original topics): weighted d 2.42 [2.33, 2.51] babies born; 2.23 [2.14, 2.32] Mt. Everest; 1.79 [1.71, 1.87] Chicago; 1.17 [1.09, 1.25] distance to NYC; original pooled 0.93 [.51, 1.33]; proportion of samples significant in the original direction 1.00 on every item; N 5,284 to 5,627 per item. Qualifies: yes",
   "quote": "\"For the purposes of the replication we provided anchoring information before asking just for the estimated quantity for four of the topics from the original study\" [...] \"Anchoring – babies born 0.93 .51, 1.33 2.43 2.60 2.41, 2.79 2.42 2.33, 2.51 0.00 1.00 0.00 t = 90.49 5,607 5,609 <.001\" [...] \"Anchoring – Mt. Everest 0.93 .51, 1.33 2.00 2.45 2.12, 2.77 2.23 2.14, 2.32 0.00 1.00 0.00 t = 83.66 5,625 5,627 <.001\" [...] \"Anchoring – Chicago 0.93 .51, 1.33 1.88 2.05 1.84, 2.25 1.79 1.71, 1.87 0.00 1.00 0.00 t = 65.00 5,282 5,284 <.001\" [...] \"Anchoring – distance to NYC 0.93 .51, 1.33 1.18 1.27 1.13, 1.40 1.17 1.09, 1.25 0.00 1.00 0.00 t = 42.86 5,360 5,362 <.001\" [...] \"The Anchoring original effect size is a mean point-biserial correlation computed across 15 different questions in a test-retest design, whereas the present replication adopted a between-subjects design with random assignments.\"",
   "removal_reason": "",
   "source": "As B5r.",
   "source_url": "https://osf.io/ebmf8/",
   "locator": "Anchoring description p. 144; Table 2 and notes p. 148; note: 36 samples (B5r); original CI 95%, replication CI 99% (Table 2 columns)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B7r",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B7r",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "SSRP replication of Karpicke & Blunt (pre-registered on OSF before data collection; one team): replication r = 0.384, p = 0.006, n = 49, against original r = 0.602, p < 0.001, n = 40; relative effect size 0.638; hard copies of the original materials. Qualifies: yes",
   "quote": "\"pre-registered at the Open Science Framework (OSF, https://osf.io/pfdyw/) prior to starting the data collection\" [...] \"In a memory test one week after learning, Retrieval Practice leads to participants recalling more correct information than Concept-Mapping.\" [...] \"Original test statistics: N = 40 (20 in each condition)\" [...] \"Karpicke and Blunt (2011), Science 0 . 602 < 0. 001 40 (40) 0 . 384 0 . 006 49 (49) yes 0 . 922 0 . 638\" [...] \"our replication used hard copies of all of the original materials used in the original study\" [...] \"Karpicke and Blunt (2011), Science ✓ ✗ ✓\" [decoded Supplementary Table 3 row: original r = 0.602, p < 0.001, n = 40; replication r = 0.384, p = 0.006, n = 49; relative effect size 0.638; the text layer spaces the digits]",
   "removal_reason": "",
   "source": "Camerer et al. (2018), Nature Human Behaviour 2, 637–644, Supplementary Information (published SI file from Nature's static-content server).",
   "source_url": "https://static-content.springer.com/esm/art%3A10.1038%2Fs41562-018-0399-z/MediaObjects/41562_2018_399_MOESM1_ESM.pdf",
   "locator": "SI p. 9 (software); p. 11 (pre-registration); p. 50 (hypothesis); Supp. Table 2 p. 53; Supp. Table 3 p. 54",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "B8r",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-B8r",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Many Labs 1, gain vs loss framing: original d = 1.13 [.89, 1.37]; replication weighted d = 0.60 [0.53, 0.67], unweighted 0.62 [0.52, 0.71]; chi-square = 516.4, N = 6,271, p < .001; proportion of samples significant in the original direction 0.86. Qualifies: yes",
   "quote": "\"Gain vs. loss framing 1.13 .89, 1.37 0.58 0.62 0.52, 0.71 0.60 0.53, 0.67 0.00 0.86 0.14 v 2 = 516.4 1 6,271 <.001\" [...] \"The replication replaced the phrase ‘‘the United States ’’ with the country of data collection, and the word ‘‘Asian’’ was omitted from\"",
   "removal_reason": "",
   "source": "As B5r.",
   "source_url": "https://osf.io/ebmf8/",
   "locator": "Framing description p. 144; Table 2 p. 148; note: 36 samples (B5r); original CI 95%, replication CI 99% (Table 2 columns)",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F1",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F1",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Our test, H1: first results at p < .001 held 58% (589 re-runs, 294 papers); .01 to .05 held 38% (341, 189); +20 points [+12, +28]. Scope: replications people chose to run, judged by the replicating authors",
   "quote": "HEADLINE H1: original p < .001 replicated 58% (589 replications, 294 papers); .01 to .05 replicated 38% (341, 189 papers); difference +20 points [95% CI +12 to +28] → CUE HOLDS",
   "removal_reason": "",
   "source": "fred_two_cues.py, pre-registered (plan, script and results on the methods page, durabilitycurve.com/methods/before-you-trust-a-study/); FReD (Röseler et al. 2024, CC BY 4.0), OSF 2tbvd v1",
   "source_url": "https://durabilitycurve.com/methods/before-you-trust-a-study/",
   "locator": "",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F6",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F6",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Anchoring original's strength, via FReD's quotation of the paper (full text not obtained): t(102) = 7.00, p < .01 as reported; the statistic itself implies a p far below the reported threshold",
   "quote": "There was a highly significant difference between individual averages of the transformed estimates made with high and low anchors, t(102) = 7.00, p < .01.",
   "removal_reason": "",
   "source": "FReD claim_text_o for doi 10.1177/01461672952111004 (secondary: a database quoting the paper)",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F7",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F7",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Rule of thumb for the card's statistic fallback: with at least 30 degrees of freedom, t above 3.646 (or F(1, df) above 13.29) means two-tailed p below .001; critical t is 3.551 at 40 df, 3.46 at 60 df, 3.3 at 1000 df",
   "quote": "30 3.646 F(1,df)= 13.29 / 40 3.551 F(1,df)= 12.61 / 60 3.46 F(1,df)= 11.97 / 1000 3.3 F(1,df)= 10.89",
   "removal_reason": "",
   "source": "own run: scipy.stats.t.ppf(1 - 0.0005, df), scipy 1.13.1",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F12",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F12",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "A no-sign failure from the 2015 sample: LoBue & DeLoache 2008 (attention to snakes and other fear-relevant images) was a strong pass (p = 0.000665379), a main effect, rated 2 for surprise (not surprising); a re-run of the same size (48) found no reliable effect (p = 0.237041962; replicators' verdict: no)",
   "quote": "Study Title (O) = Detecting the snake in the grass: attention to fear-relevant stimuli by adults and young children. / Type of effect (O) = main effect / Surprising result (O) = 2 / T_pval_USE..O. = 0.000665379 / N (O) = 48 / N (R) = 48.0 / T_pval_USE..R. = 0.237041962 / Replicate (R) = no",
   "removal_reason": "",
   "source": "OSC 2015 public data, rpp_data.csv, Study Num 161 (OSF 55ddc2248c5e4a3ef7c0e4b7); own read of the row",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F13",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F13",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Our held-out test: on 27 SSRP and Many Labs 2 findings not in the 2015 sample, with lay readers' doubt as the surprise sign, act held 5/5, try small 9/18 (50%), wait 0/4; act - wait +100 points, Fisher p = 0.00794",
   "quote": "Held-out card test · 27 findings (SSRP + Many Labs 2), signs frozen before outcomes / by action: act 5/5 (100%) · try small 9/18 (50%) · wait 0/4 (0%) / act - wait +100 points, Fisher p = 0.00794, monotone = True -> CARD SORTS ON HELD-OUT CLAIMS",
   "removal_reason": "",
   "source": "heldout_card_test.py, pre-registered (plan, script and results on the methods page, durabilitycurve.com/methods/before-you-trust-a-study/); Hoogeveen et al. 2020 data (osf.io/wg9s3)",
   "source_url": "https://durabilitycurve.com/methods/before-you-trust-a-study/",
   "locator": "",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F14",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F14",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Held-out comparisons: base rate held 52% (14/27); p-value cue alone 67%; lay majority alone 67%; card act-vs-wait 100% on 9 of 27 decided",
   "quote": "accuracy, p-value only (strong -> holds): 67% (n = 27) / accuracy, lay majority only (most predict it replicates -> holds): 67% (n = 27) / accuracy, card act vs wait (try small abstains): 100% on 9 of 27 decided / base rate held: 52% (14/27)",
   "removal_reason": "",
   "source": "as F13",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-26",
   "in_essay": ""
  },
  {
   "id": "F15",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F15",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Our small-group check: with sign 4 judged by one lay rater instead of the crowd (F13), the card still sorted the held-out findings: act held 100%, try small 52%, wait 32%; median act - wait +70; the held-out rule held in 82% of draws (88% with three raters, 90% with five)",
   "quote": "k = 1 panel: agree w/ crowd 67% · act 100% (avg 3.6) · try 52% · wait 32% (avg 9.0) · median gap +70 [+42, +100] · rule holds 82% / -> SMALL GROUP KEEPS THE SORT (k = 1) / k = 3 panel: ... rule holds 88% / k = 5 panel: ... rule holds 90%",
   "removal_reason": "",
   "source": "smallgroup_check.py, pre-registered (plan, script and results on the methods page, durabilitycurve.com/methods/before-you-trust-a-study/); Hoogeveen et al. 2020 data (osf.io/wg9s3)",
   "source_url": "https://durabilitycurve.com/methods/before-you-trust-a-study/",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F17",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F17",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Exact 95% intervals for the held-out cells: act 5 of 5, 47.8% to 100.0%; wait 0 of 4, 0.0% to 60.2%; try small 9 of 18, 26.0% to 74.0%",
   "quote": "5/5: exact 95% [47.8%, 100.0%] / 0/4: exact 95% [0.0%, 60.2%] / 9/18: exact 95% [26.0%, 74.0%]",
   "removal_reason": "",
   "source": "Clopper–Pearson intervals computed with scipy.stats.beta from F13's counts",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "A01h",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-A01h",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "In the 2015 project, each original's \"surprising result\" score was the average rating of at least 3 volunteers, naive to the results, on a 1 to 6 scale",
   "quote": "\"Average response of at least 3 volunteers, naive to results, who report how suprising they found the key effect to be: 1: not at all, 2: slightly, 3: somewhat, 4: moderately, 5: very, 6: extremely\" [sic: \"suprising\"]",
   "removal_reason": "",
   "source": "OSC 2015 public data codebook, rpp_data_codebook.csv (OSF 55ddc2248c5e4a3ef7c0e4b7)",
   "source_url": "",
   "locator": "row \"Surprising result (O)\", column Description",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "CR1a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-CR1a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Inputs include the original study's sample size, effect size, p-value, and whether the tested effect is an interaction",
   "quote": "\"among these features we have sample size, the effect size and p-value originally measured, and whether a finding is an effect of one variable or an interaction between multiple variables\"",
   "removal_reason": "",
   "source": "Altmejd, Dreber, Forsell, Huber, Imai, Johannesson, Kirchler, Nave & Camerer (2019), \"Predicting the replicability of social science lab experiments\", PLOS ONE 14(12): e0225826. Publisher PDF + JATS XML (open access)",
   "source_url": "https://doi.org/10.1371/journal.pone.0225826",
   "locator": "§ 1 Introduction, PDF p. 2 (PDF loose match: \"vari- able\" breaks across a line; XML strict)",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "CR3a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-CR3a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "They proposed moving the default significance threshold from 0.05 to 0.005 for claims of new discoveries",
   "quote": "\"We propose to change the default P-value threshold for statistical significance from 0.05 to 0.005 for claims of new discoveries.\"",
   "removal_reason": "",
   "source": "Benjamin et al. (2018), \"Redefine statistical significance\", Nature Human Behaviour 2: 6–10 (online 1 Sep 2017). Publisher page, abstract only: the full text is paywalled",
   "source_url": "https://www.nature.com/articles/s41562-017-0189-z",
   "locator": "Abstract (loose match: the page's markup renders \"P -value\")",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F20",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F20",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Our SCORE test, pre-registered: on SCORE's re-runs (331 claims, 212 papers), originals at p < .001 replicated 66% against 34% for p between .01 and .05, a gap of +32 points (+16 to +46), each paper weighted once: the cue held on the large batch our falsifier named",
   "quote": "A_S_vs_W: 66% (152 claims, 96 papers) vs 34% (63, 48) · +32 [+16, +46] -> CUE HOLDS / SCORE cue test · mode real · 331 claims from 212 papers",
   "removal_reason": "",
   "source": "score_cue_test.py, pre-registered (plan, script and results on the methods page, durabilitycurve.com/methods/before-you-trust-a-study/); SCORE data, OSF dtzx4 (CC0)",
   "source_url": "https://durabilitycurve.com/methods/before-you-trust-a-study/",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F21",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F21",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "SCORE, the card's first sign as a reader applies it: strong passes replicated 65% against 38% for weak passes, +27 (+14 to +39); psychology and health only, p-band test +25 (+1 to +47); middle band .001 to .01 held 22%",
   "quote": "B_card_strong_vs_weak: 65% (175 claims, 113 papers) vs 38% (144, 105) · +27 [+14, +39] -> CUE HOLDS / [psychology and health (secondary)] A_S_vs_W: 67% (54 claims, 45 papers) vs 42% (33, 23) · +25 [+1, +47] / A middle band (.001-.01) rate: 22%",
   "removal_reason": "",
   "source": "as F20",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F22",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F22",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "SCORE, sample size: the biggest third of original samples replicated 52% against 40% for the smallest third, +12 (−5 to +27), not detectable, so the sample-size falsifier did not fire",
   "quote": "C_size_top_vs_bottom: 52% (98 claims, 57 papers) vs 40% (98, 72) · +12 [-5, +27] -> NO DETECTABLE ADVANTAGE FOR BIG SAMPLES / falsifier, sample size: not fired",
   "removal_reason": "",
   "source": "as F20",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "W1a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-W1a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "SCORE (Tyner et al., Nature 2026) re-ran 274 claims of positive results from 164 papers published 2009 to 2018 in 54 social- and behavioural-science journals; median power 99.6% to detect the original effect; 151 of 274 replicated (55.1%)",
   "quote": "\"Here we attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioural sciences. Replications were high powered on average to detect the original effect size (median of 99.6%)\" … \"Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1%\"",
   "removal_reason": "",
   "source": "Tyner et al. (2026), \"Investigating the replicability of the social and behavioural sciences\", Nature, DOI 10.1038/s41586-025-10078-y; abstract from the Nature page",
   "source_url": "https://www.nature.com/articles/s41586-025-10078-y",
   "locator": "Abstract",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "W1b",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-W1b",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "SCORE's papers came from a stratified random sample, but which claims were re-run depended on feasibility, so the re-run set is not itself random",
   "quote": "\"random sampling is lost because selection is based on feasibility, available resources, and available expertise.\"",
   "removal_reason": "",
   "source": "Tyner et al. (2026), accepted manuscript (JYX repository)",
   "source_url": "https://jyx.jyu.fi/handle/123456789/109872",
   "locator": "Methods, \"Sampling and selecting claims\", PDF p. 18",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "W1c",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-W1c",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Colquhoun (2014) argued that to keep false discoveries below 5% you should insist on p ≤ 0.001",
   "quote": "\"It is concluded that if you wish to keep your false discovery rate below 5%, you need to use a three-sigma rule, or to insist on p≤0.001.\"",
   "removal_reason": "",
   "source": "Colquhoun (2014), \"An investigation of the false discovery rate and the misinterpretation of p-values\", Royal Society Open Science 1:140216",
   "source_url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC4448847/",
   "locator": "Abstract",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "W1d",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-W1d",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Gordon et al. (2021): across findings from four replication projects (RPP, EERP, ML2, SSRP), originals at p ≤ 0.005 replicated about 74% against 28% above it",
   "quote": "\"While the replication rate for findings with p < = 0.005 is about 74%, the replication rate for findings with p > 0.005 drops to 28%.\"",
   "removal_reason": "",
   "source": "Gordon, Viganola, Dreber, Johannesson & Pfeiffer (2021), PLOS ONE 16: e0248780 (RPP 40, EERP 18, ML2 24, SSRP 21)",
   "source_url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC8046229/",
   "locator": "§ 3.5 \"P value analysis\"",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "W1f",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-W1f",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Spencer Greenberg, in a podcast interview: in 325 replications, originals at p ≤ 0.01 replicated about 72%, against about 48% above 0.01",
   "quote": "\"we looked at 325 studies that were replications of existing studies.\" … \"when the original study p-value was at most 0.01 — so it was 0.01 or less — about 72% replicated.\" … \"when the p-value is greater than 0.01, only about 48% replicated.\"",
   "removal_reason": "",
   "source": "80,000 Hours podcast episode #147, Spencer Greenberg, transcript",
   "source_url": "https://80000hours.org/podcast/episodes/spencer-greenberg-stopping-valueless-papers/",
   "locator": "Transcript § \"How often papers replicate with different p-values [00:43:31]\"",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F26",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F26",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "Forced to call every one of the 27 held-out findings act or wait, the card scores 18 of 27, the same share as the p-value alone: its act calls (5 of 5) and wait calls (4 of 4) are right, and the 18 try-small findings split 9 held, 9 not, so either forced call gets 9",
   "quote": "forced: act 5 right + wait 4 right + 9 of 18 try-small either way = 18 of 27 = 66.7",
   "removal_reason": "",
   "source": "arithmetic on F13's counts",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F28",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F28",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "EXPLORATORY: in SCORE's new-data replications only (146 claims, 114 papers), originals at p < .001 replicated 67% against 32% for .01 to .05, +35 (+13 to +55)",
   "quote": "EXPLORATORY · new-data replications only · 146 claims, 114 papers / A S vs W: 67% (51, 46) vs 32% (39, 32) · +35 [+13, +55] -> CUE HOLDS",
   "removal_reason": "",
   "source": "explore_score_newdata.py (exploratory, added after the locked run)",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F29",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F29",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "In the held-out test, all 5 strong passes were act calls; the other two signs split the 22 weak passes: 9 of 18 with no other sign held, 0 of 4 with one did",
   "quote": "weak passes 22 by action {'try small': 18, 'wait': 4} strong passes 5 strong actions {'act': 5} / by action: act 5/5 (100%) · try small 9/18 (50%) · wait 0/4 (0%)",
   "removal_reason": "",
   "source": "heldout_signs.csv and F13's counts",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "W1g",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-W1g",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "A SCORE companion paper (Abatayo et al., MetaArXiv, 2026) reports the original p-value's correlation with replicability as -.11 (-.26 to .04) by paper and -.18 (-.29 to -.05) by claim; the Nature paper defers to it",
   "quote": "\"Original p-value\" … \"-.11\" … \"[-.26, .04]\" (Table 3) … \"Original p-value\" … \"-.18\" … \"[-.29, -.05]\" (Table S19, by claim) … \"We did not explore correlations with replication outcomes\"",
   "removal_reason": "",
   "source": "Abatayo et al. (2026), \"Assessments of Credibility in the Social and Behavioral Sciences\", MetaArXiv 7u58q_v1; Tyner et al. (2026) accepted manuscript for the deferral",
   "source_url": "https://osf.io/preprints/metaarxiv/7u58q_v1",
   "locator": "Table 3, p. 9; Table S19; Nature accepted manuscript",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R4d",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R4d",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Alvaro de Menard (2020), arguing from replication-forecast data, put replication below 50% for originals with p > .001 and about 10 percentage points lower for interaction effects",
   "quote": "\"The base replication rate for studies with p>.001 is below 50%.\" … \"A paper whose main finding is an interaction effect is about 10 percentage points less likely to replicate.\"",
   "removal_reason": "",
   "source": "de Menard (2020), \"What's Wrong with Social Science and How to Fix It\", Fantastic Anachronism, 11 Sep 2020 (basis: Replication Markets forecasts)",
   "source_url": "https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/",
   "locator": "Sections \"What To Do\" and \"Interaction Effects Bad\"",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R4g",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R4g",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "SCORE's 274 claims split into 128 new-data and 146 secondary-data replications; secondary-data replications used existing data independent of the original",
   "quote": "\"128 / 128\" … \"146 / 146\" … \"Secondary data replications are those that used existing data that was independent of the original investigation.\" (274 = 128 + 146)",
   "removal_reason": "",
   "source": "Tyner et al. (2026), accepted manuscript, Table 4",
   "source_url": "https://jyx.jyu.fi/handle/123456789/109872",
   "locator": "Table 4 and note",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Vu & Faridani forecast replication from each original's p-value, together with an estimate of how large effects in that field tend to be, in three projects: economics (Camerer 2016), psychology (OSC 2015) and social science (Camerer 2018)",
   "quote": "\"We estimate the predictive power curve in three empirical literatures with systematic replication evidence: experimental economics (Camerer et al., 2016), psychology (Open Science Collaboration, 2015), and experimental social science (Camerer et al., 2018).\"",
   "removal_reason": "",
   "source": "Vu & Faridani (2026), \"How Replicable Are Statistically Significant Findings?\", arXiv 2608.23257 (working paper)",
   "source_url": "https://arxiv.org/abs/2608.23257",
   "locator": "p. 2",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5b",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5b",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "On the 70 findings that also had prediction-market forecasts, the p-only forecasts' average squared error (Brier score) was 0.216 against the markets' 0.220, pooled and weighted",
   "quote": "\"the validation exercise uses the common sample of 70 studies for which both predictions are available\" … \"In the weighted pooled sample, predictive power slightly outperforms prediction markets, with Brier scores of 0.216 and 0.220, respectively.\"",
   "removal_reason": "",
   "source": "Vu & Faridani (2026), \"How Replicable Are Statistically Significant Findings?\", arXiv 2608.23257 (working paper)",
   "source_url": "https://arxiv.org/abs/2608.23257",
   "locator": "pp. 11–12, Table 1 (psychology alone 0.243 and 0.243)",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5d",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5d",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "In psychology, a finding at p = 0.05 has an expected replication probability of 0.22 in a same-size re-run; across the three fields, 0.1 to 0.25",
   "quote": "\"Evaluating the curve at the 5% significance threshold, we find that replicability ranges between 0.1 and 0.25 across fields.\" … Figure 2, psychology panel: \"(0.05, 0.22)\"",
   "removal_reason": "",
   "source": "Vu & Faridani (2026), \"How Replicable Are Statistically Significant Findings?\", arXiv 2608.23257 (working paper)",
   "source_url": "https://arxiv.org/abs/2608.23257",
   "locator": "p. 13 and Figure 2, p. 14",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5f",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5f",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Carter & McCullough re-analysed the 198 published tests in Hagger et al.'s 2010 meta-analysis",
   "quote": "\"Hagger et al. (2010) reported a meta-analysis of the 198 published tests of the depletion effect\"",
   "removal_reason": "",
   "source": "Carter & McCullough (2014), Frontiers in Psychology 5:823",
   "source_url": "https://www.frontiersin.org/articles/10.3389/fpsyg.2014.00823/full",
   "locator": "Introduction",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5g",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5g",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Corrected for small-study effects (publication bias among them), the depletion effect was not meaningfully different from zero",
   "quote": "\"an indication that the depletion effect is actually no different from zero\" … \"after controlling for the influence of small-study effects, our results do not support the claim that the depletion effect is meaningfully different from zero\"",
   "removal_reason": "",
   "source": "Carter & McCullough (2014), Frontiers in Psychology 5:823",
   "source_url": "https://www.frontiersin.org/articles/10.3389/fpsyg.2014.00823/full",
   "locator": "Abstract; Discussion",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5h",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5h",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Carter & McCullough turned to the meta-analysis after failing to produce the depletion effect in their own work",
   "quote": "\"In response to our inability to produce the basic depletion effect in our own work, we turned to Hagger et al.'s (2010) meta-analysis\"",
   "removal_reason": "",
   "source": "Carter & McCullough (2014), Frontiers in Psychology 5:823",
   "source_url": "https://www.frontiersin.org/articles/10.3389/fpsyg.2014.00823/full",
   "locator": "Introduction",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5j",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5j",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Falkovich's lay rules include anti-significance (a result at p = 0.049 is suspect) and Taleb's grandma (an effect that sounds implausible probably is not real)",
   "quote": "\"If a study has p=0.049 it is fake.\" … \"If the purported effect sounds implausible, it is.\"",
   "removal_reason": "",
   "source": "Jacob Falkovich (7 Sep 2018), \"The Scent of Bad Psychology\", Putanumonit",
   "source_url": "https://putanumonit.com/2018/09/07/the-scent-of-bad-psychology/",
   "locator": "Rules of Anti-Significance and Taleb's Grandma",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R5k",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R5k",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Bailey's rules of thumb include trusting overall effects over interaction or subgroup effects, extraordinary claims needing extraordinary evidence, and the real effect being half what the paper claims; they do not use p-values",
   "quote": "\"Overall effects are more trustworthy than interaction or subgroup effects\" … \"extraordinary claims require extraordinary evidence\" … \"The real effect size is half what the paper claims\"",
   "removal_reason": "",
   "source": "James Bailey (2 Apr 2026), \"How Much To Trust Research Papers? My Rules Of Thumb\", Economist Writing Every Day",
   "source_url": "https://economistwritingeveryday.com/2026/04/02/how-much-to-trust-research-papers-my-rules-of-thumb/",
   "locator": "rules 2, 6 and 8",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R6a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R6a",
   "status": "REPORTED",
   "status_label": "Reported",
   "removed": false,
   "claim": "Lindsay's \"troubling trio\" (2015): low statistical power, a surprising result, and a p-value only slightly lower than .05",
   "quote": "\"Described as a combination of low statistical power, a surprising result, and a p-value only slightly lower than .05.\" (FORRT glossary entry, citing Lindsay 2015)",
   "removal_reason": "",
   "source": "Lindsay (2015), \"Replication in Psychological Science\", Psychological Science 26(12), 1827–1832",
   "source_url": "https://forrt.org/glossary/english/the_troubling_trio/",
   "locator": "FORRT glossary",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R6b",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R6b",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "SCORE reports thirteen measures of replication success, whose estimates range from 28.6% to 74.8% (median 49.3%)",
   "quote": "\"Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median of 49.3%).\"",
   "removal_reason": "",
   "source": "Lakens (3 May 2026), \"There are 13 ways to analyse a replication study, but only one of them is coherent\", The 20% Statistician",
   "source_url": "https://daniellakens.blogspot.com/2026/05/there-are-13-ways-to-analyse.html",
   "locator": "quoting Tyner et al. 2026",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R7a",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R7a",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "The 1998 radish experiment was repeated once, as a school science-fair project, reported in the original paper's footnote; we found no registered re-run of that experiment",
   "quote": "\"As this article went to press, we were notified that this experiment had been independently replicated by Timothy J. Howe, of Cole Junior High School in East Greenwich, Rhode Island, for his science fair project\"",
   "removal_reason": "",
   "source": "Baumeister, Bratslavsky, Muraven & Tice (1998), JPSP 74(5), footnote 2",
   "source_url": "https://faculty.washington.edu/jdb/345/345%20Articles/Baumeister%20et%20al.%20%281998%29.pdf",
   "locator": "footnote 2",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R7c",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R7c",
   "status": "REPORTED",
   "status_label": "Reported",
   "removed": false,
   "claim": "A 2026 re-analysis of SCORE's data finds that replication-success methods cannot reliably reach a conclusion at typical sample sizes",
   "quote": "\"Replication success methods are unable to reliably draw a conclusion about replication success for typical sample sizes of the original study and replication\"",
   "removal_reason": "",
   "source": "van Aert, van Assen & López-Nicolás (2026), PLOS ONE 21(9): e0358126",
   "source_url": "https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0358126",
   "locator": "Abstract",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R7e",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R7e",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "A pre-registered 2023 study (N = 230) reproduced the retrieval-practice advantage on recall (verbatim) questions, then found no significant difference when the concept-mapping group also memorised (p = 0.854); the authors call the advantage a methodological artifact",
   "quote": "\"While we replicated the previous finding that retrieval practice in combination with additional memorizing produces more learning than concept mapping without additional memorizing, this advantage of retrieval practice over concept mapping vanished when participants in the concept mapping condition, too, memorized\" … \"performance did not differ ( p = 0.854) between the RP + AM + AI condition and the CM + AM + AI condition\" … \"resulting in a final sample size of N = 230\" … \"followed our preregistration at Open Science Foundation\"",
   "removal_reason": "",
   "source": "Mayrhofer, Kuhbandner & Frischholz (2023), Frontiers in Psychology 14:1258359",
   "source_url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC10783554/",
   "locator": "Abstract; Methods",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "R7f",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-R7f",
   "status": "VERIFIED",
   "status_label": "Verified",
   "removed": false,
   "claim": "Hoogeveen et al. reported that findings their lay participants, as a group, predicted would fail did fail to replicate",
   "quote": "\"when participants as a group predicted that a particular effect would fail to be replicated, it indeed failed to be replicated\"",
   "removal_reason": "",
   "source": "Hoogeveen, Sarafoglou & Wagenmakers (2020), AMPPS",
   "source_url": "https://doi.org/10.1177/2515245920919667",
   "locator": "Results",
   "date_read": "2026-10-27",
   "in_essay": ""
  },
  {
   "id": "F30",
   "url": "https://durabilitycurve.com/claims/survives-a-recheck/#c-F30",
   "status": "EXECUTED",
   "status_label": "Executed",
   "removed": false,
   "claim": "In the held-out test, the card's four wait calls are exactly the four findings with the lowest lay share predicting replication (Lee & Schwarz 0.172, Gervais & Norenzayan 0.299, Shah et al. 0.368, Tversky & Gati 0.468); all four are weak passes, one has an interaction",
   "quote": "Lee & Schwarz (2010) lay 0.172 wait · Gervais & Norenzayan (2012) lay 0.299 wait · Shah et al. (2012) lay 0.368 wait · Tversky & Gati (1978) lay 0.468 wait · next lowest Alter et al. (2007) lay 0.504 try small; doubted True only on the four; interaction True on one",
   "removal_reason": "",
   "source": "heldout_signs.csv",
   "source_url": "",
   "locator": "",
   "date_read": "2026-10-27",
   "in_essay": ""
  }
 ]
}