Proof & Trust 15 of 15 6 min read

A Fake Statistics Site Cost $50. ChatGPT Kept Citing It.

Nine of the 11 AI numbers that failed our check gave it away on the page the AI linked. A second AI flagged most of the ones that traced, too.

Harry Floyd 6 min read

How this is checked 68 claims · 57 verified at the source · last checked 11 Nov 2026 The claim ledger 57 of 68 claims
verified at the source
Checked 11 Nov 2026 How this is checked
Rests on
Law IV, Instruments Over Theory. It has been put to a real test and amended or narrowed under that pressure. Last tested 21 Jun 2026.
What would prove it wrong
  1. If a larger test finds that a second AI's fact-check flags at least 70% of bad statistics, and flags good ones at no more than half that rate, then the piece's advice on fact-checking is wrong.
  2. If most AI-supplied statistics that fail do so in ways the cited page can't show (no drift visible), then "check the cited page first" is the wrong lead step.

On the track record, not yet testable2 untestedThe re-check

Evidence
68 claims: 57 verified at the primary source, 10 from runs made for the piece and 1 reported.
Contents · 6 sections
  1. 01How often the page and the count come apart
  2. 02What we found in 40 slide statistics
  3. 03A second AI, asked to check
  4. 04The Number Check
  5. 05What this shows, and what it doesn’t
  6. 06The page in the journal’s references

In March 2026, Metehan Yesilyurt, who now works at Peec AI, a firm whose tool tracks how brands show up in AI search, built a website of statistics. In a single day, using GPT-4.1, he generated 50,000 pages of figures on marketing, e-commerce and web technology for 200 countries. He later added 15,000 listicles, and the whole experiment cost about $50. Every number was invented. In his words, “I never checked a single one.”

Google buried the site. ChatGPT kept citing it. His analytics show more than 10,000 sessions from ChatGPT since March, and on the prompts he tracks in his firm’s own tool, the site ranked second for visibility, behind only Statista. In the 30 days before his October write-up, ChatGPT also requested its pages about 72,000 times, each one “fetching a fake statistics page to answer a real question”.

The numbers travelled. An article in an academic journal, on live-stream shopping, lists StateGlobe’s “Live Streaming Growth and Video Marketing Statistics in China” among its references. Yesilyurt also posted a screenshot of a market report from Qatar’s largest financial group that quotes the site: “According to StateGlobe, as much as 68% of e-commerce orders are delivered within the same day.”

Cover art for A Fake Statistics Site Cost $50. ChatGPT Kept Citing It.

A link to a page like these passes the click-the-link check: it loads, and the number is on it. The question it fails is who counted it.

A number can also go wrong when someone did count it. It slips on the way to you, and arrives with the wrong year, place, group or measure, or behind a link that doesn’t show it. We traced 40 statistics that two AI assistants gave us for slides. None of them led to the fake site, and most of the ones that failed had slipped in this second way.

The short version: In our test of 40 statistics that AI gave for slides, 25 of the 39 we could judge traced to whoever counted them, and 9 of the 11 that failed our check showed it on the page the AI linked. A second AI asked to fact-check flagged all 11, and most of the ones that traced. Four questions show where to look.

How often the page and the count come apart

Trakkr’s study, published on 28 October by a company that sells tools for getting brands cited in AI answers, is the larger and earlier test of where AI’s numbers come from. It asked ChatGPT, Perplexity, Gemini, Claude and Google’s AI Overviews 200 questions that call for a number, then followed 1,285 of the statistics they cited back, page by page.

The cited page had the figure 82% of the time. Four in ten trails reached a study that said the same thing. Another 8% reached the study but found the number or its context had changed on the way. Another four in ten stopped at a source Trakkr couldn’t check, and 11% led nowhere: every page repeated the number and credited no one.

An AI model judged the trails, and people re-checked 150 of them. One trail went from “94% of first impressions are design-related” back to a 2004 study in which 15 British women judged health websites.

What we found in 40 slide statistics

We asked two AI assistants with web search, Claude Sonnet 5.5 and GPT-5.6 Sol, run through their command-line developer tools, the same 20 questions, each ending: “What’s a good recent statistic on X? Give me the number and a link to the source.” We locked the rules for judging the answers, then collected them.

Two independent traces, each run by AI agents given those rules and the answers with the assistants’ names removed, followed every number back and recorded each page they opened, so any call could be checked. They agreed on 37 of the 40.

25 of 39 traced back to whoever counted them, with the same figure, and one more couldn’t be judged. Of the 14 that didn’t trace, three named a counter whose original the tracers couldn’t open. The other 11 failed our check, and 10 of those had slipped. In nine, the page the AI linked gave it away: the number wasn’t on it, or it didn’t describe the same thing.

Two of the fourteen:

Two statistics that did not trace: what the AI said, and what the source says. AI: 19% of Polish microbusinesses invest in SEO and content marketing. Source: 19% of the 206 firms that advertise regularly, the only firms asked that question. AI: 71% of B2B buyers said they consumed blog content, from Demand Gen Report's 2018 survey. Source: the cited page is the 2014 survey, with no 71% on it; the 2018 report does have it.

A second AI, asked to check

Asking another AI to fact-check a number feels like a second opinion, so we tried one prompt on both assistants. Each checked the other’s numbers, given the source link, and ended on one of three verdicts: safe to use, don’t use as is, or could not verify.

It said “don’t use as is” or “could not verify” to all 11 numbers that failed our check, and to 17 of the 25 that traced. All eight of its “safe to use” verdicts fell on numbers that traced, and eight cases are too few to say how often a “safe” would hold. A warning from it didn’t tell you which number to drop, because two in three numbers that traced got one too.

Where it helped was finding sources: it turned up the 2018 report behind the 71% blog figure. Use it that way, then read the source yourself.

The Number Check

1. Is the number on the page? Find the exact figure where the link lands. If it isn’t there, you haven’t checked it yet, even when the number is real, like the 71% above.

2. Is it the same thing? Read the sentence the number sits in, and any small print, and check the year, the place, the group and what exactly was measured. Each slip in the table fails one of those, or the first question.

3. Who counted it, and how? Follow the number back until you reach whoever ran the survey, kept the register or took the measurement: what Mike Caulfield’s SIFT method calls tracing a claim “back to the original context”. Look for who was asked, how many, and when. A page that only lists numbers didn’t count anything, and the 94% about websites came from a study of 15 women.

4. Can you say it their way? Write the sentence the way the counter would. “71% of B2B buyers said they consumed blog content” becomes “In Demand Gen Report’s 2018 Content Preferences Survey, 71% of respondents said they reviewed a blog while on their buying journey”, linked to that report. If you can’t reach anyone who counted it, leave it out, or say where you found it and that you couldn’t trace it.

What this shows, and what it doesn’t

This was a small test: two assistants, twenty questions and one fact-check prompt, used through developer tools rather than the ChatGPT or Claude apps. Trakkr’s study used other questions and methods, so its figures and ours stand on separate bases. We built the four questions from these same cases and haven’t yet tested how many slips readers catch with them.

Our tracers, and our checks on them, were AI agents too. Each recorded its trail, which is what let us check them: when we re-checked ten of their calls at the source, we agreed on seven, and a third reader settled the other three, two of them in the tracers’ favour.

In Trakkr’s larger study, the cited page usually had the number while fewer than half the trails reached a matching study, which is why question 3 asks who counted it. The rules, scores, both traces, and the result that would change our advice on second opinions are on the methods page.

The page in the journal’s references

Here is the StateGlobe page with that title as it stands today. The journal’s authors accessed it on 2 March; this one was last reviewed in August. Question 1 passes: 650 million active live-stream viewers is on the page. Question 2 turns up the page’s own small print, which calls its figures 2026 estimates from compound growth modelling that may differ from actual outcomes.

Question 3 fails. The methodology box names the ITU, the World Bank and Statista only as inputs to its “projections”, and the man who built the site says every number on it was invented and that he never checked a single one.

Before a number goes on your slide, find it on the page, read the sentence and small print around it, then find whoever counted it.

Get the Number Check as a one-page card, free.

The newsletter · Free

Subscribe to get each new piece in your inbox, with something to use in every one.

Several essays a week, plus Notes through the week.

Signup completes on Substack. Unsubscribe from any email.

104 pieces 1,117 claims verified 7 instruments