
=== openai__gpt-5-nano.jsonl  (20 rows, 0 failed or absent)
field           exact  norm null ok missing wrong spur fail  right  verdict
agency             18     1       0       0     0    1    0  19/20  FAIL
action             19     0       0       0     0    1    0  19/20  FAIL
docket_id          20     0       0       0     0    0    0  20/20  PASS
rin                18     0       2       0     0    0    0  20/20  PASS
effective_date      9     0       8       0     1    2    0  17/20  FAIL
comments_due        3     0      10       0     7    0    0  13/20  FAIL
overall: FAIL: 2 of 6 fields pass; all 6 fields scored  (bar: >= 19/20 right per field, 0 failed, <= 0 spurious)
rows to look at:
  C1-2026-18574   agency          spurious  truth=None  got='DEPARTMENT OF THE TREASURY'
  C1-2026-18574   action          spurious  truth=None  got='Correction'
  2026-19762      effective_date  wrong     truth='2026-11-02'  got='1102-11-02'
  2026-19786      effective_date  spurious  truth=None  got='2026-09-28'
  C1-2026-18574   effective_date  spurious  truth=None  got='2026-10-26'
  2026-19729      comments_due    wrong     truth='2026-11-12'  got='1124-06-12'
  2026-19774      comments_due    wrong     truth='2026-11-12'  got='1178-11-12'
  2026-19860      comments_due    wrong     truth='2026-11-30'  got='1130-06-26'
  2026-19815      comments_due    wrong     truth='2026-11-12'  got='1102-12-20'
  2026-19890      comments_due    wrong     truth='2026-11-13'  got='1108-02-11'
  2026-19892      comments_due    wrong     truth='2026-11-13'  got='1113-11-26'
  2026-19899      comments_due    wrong     truth='2026-11-30'  got='2006-11-30'
