How to find hallucinated sources in commercial research

A model asked for a source will produce one, and it will look right: a plausible author, a real-sounding journal, a year, a page range. A few of those references will exist. Others are real papers that don’t say what the draft claims, and the rest are invented outright. A reader who checks one and finds it fabricated stops trusting the whole document, and in commercial research that reader is often the person deciding whether to buy.

Why it happens

A model doesn’t look anything up. It produces text that resembles the citations it has seen, so a reference is a prediction of what a citation would look like rather than a record of a document that exists. More careful prompting doesn’t fix that, because the model isn’t being careless in the first place. Somebody has to check every reference against the source itself.

How AI gets references wrong

References that don’t exist. The author is real, the journal is real, but the paper was never written. They’re the easiest to catch, and the most embarrassing to miss.

Real sources that say something else. The paper exists, but the draft has attributed a finding to it that appears nowhere in it, or has turned a cautious result into a firm one.

Numbers without a home. A statistic in confident phrasing with no origin. It may have come from a real report and been rounded along the way, or there may be nothing behind it at all.

The right source with the wrong details. The paper is right, but the year, the journal, or the author list is wrong. Harmless to the argument but fatal to credibility, because it’s the kind of thing a careful reader notices first.

How to check

Open every single reference, not a sample. Search for the title in quotation marks, then for the DOI if there is one. If neither turns up the document, treat the reference as fabricated until you’ve found it.

Read the passage being cited, rather than the abstract, and confirm that the source says exactly what the draft claims it does.

Trace every number to a page. If you can’t find the page, the number comes out, or it becomes a claim in your own voice that you’re prepared to defend.

Check the details against the document itself, since other citations of the same paper are equally unreliable until verified.

Keep a record. A line per reference, saying where you found it and what you confirmed, means you can answer in a minute when a client or a reviewer asks.

What to do with what you find

Remove a fabricated reference rather than swapping in the nearest real one, because the nearest real one probably doesn’t support the claim either. If the draft made an assertion it can’t support, the assertion goes, or it’s rewritten as your own judgment and labeled as such. A shorter document whose citations all check out is worth more than a long one with a single invention in it.

What we check

Reference checking is part of every edit we do, both for format and for whether the source exists, because hallucinated references are common in AI-assisted drafts. Going a step further and confirming that each source actually backs up the argument it’s cited for is developmental work, and we quote it separately. Either way, the refined draft comes back with every unverifiable reference flagged in the margin for you to decide on.