
=== naive__openai__gpt-5-nano__as_parsed.jsonl  (20 rows, 0 failed or absent)
field           exact  norm null ok missing wrong spur fail  right  verdict
agency              9     9       0       0     1    1    0  18/20  FAIL
action             15     3       0       0     1    1    0  18/20  FAIL
docket_id          19     1       0       0     0    0    0  20/20  PASS
rin                18     0       2       0     0    0    0  20/20  PASS
effective_date      7     3       8       0     0    2    0  18/20  FAIL
comments_due        5     5       9       0     0    1    0  19/20  FAIL
overall: FAIL: 2 of 6 fields pass; all 6 fields scored  (bar: >= 19/20 right per field, 0 failed, <= 0 spurious)
rows to look at:
  2026-19743      agency          wrong     truth='Office of the Assistant Secretary for Community Planning and Development'  got='Department of Housing and Urban Development'
  C1-2026-18574   agency          spurious  truth=None  got='Internal Revenue Service'
  C1-2026-18574   action          spurious  truth=None  got='Correction'
  2026-19815      action          wrong     truth='Supplemental notice of proposed rulemaking (SNPRM)'  got='SNPRM'
  2026-19786      effective_date  spurious  truth=None  got='2026-09-28'
  C1-2026-18574   effective_date  spurious  truth=None  got='October 26, 2026'
  C1-2026-18574   comments_due    spurious  truth=None  got='October 26, 2026'
