Disclosure: This post contains affiliate links; we may earn a commission at no extra cost to you.
Independent 2025–2026 tracking of frontier models shows even strong performers like GPT-4o and Claude still hallucinate on 15–20% of factual citation tasks, and that rate climbs to 35–55% on niche or very recent topics where the model has thinner training data to draw from. Open-ended generation — the exact mode you’re in when you ask an AI to “write an article about X” — sits at the high end of that range, with some benchmarks putting hallucination rates for unconstrained generation as high as 40–80% depending on topic obscurity. Translation: if you publish an AI draft without checking it, you are publishing wrong information on purpose, not by accident.
What actually goes wrong
AI models don’t lie the way people do — they don’t know they’re wrong. The failure modes are specific and worth recognizing on sight:
- Invented statistics — a suspiciously precise number (“73.4% of marketers”) with no traceable source, because the model pattern-matched what a plausible-sounding stat looks like.
- Fabricated citations — a real-sounding study, author, or URL that either doesn’t exist or doesn’t say what the draft claims it says.
- Stale-as-current — accurate information from the model’s training data presented as today’s pricing, features, or policy when the underlying product has since changed.
- Misattributed quotes — a real quote assigned to the wrong person, or a paraphrase presented inside quotation marks as if verbatim.
A repeatable fact-check workflow
Don’t try to verify everything with equal effort — triage first, then check.
- Flag every checkable claim — any number, date, price, named study, or quote. Generic advice sentences don’t need this pass.
- Verify against a primary source, not another AI summary or a listicle that might itself be AI-generated. For pricing and specs, go to the vendor’s own page. For research claims, find the actual paper or dataset.
- Check the date on anything time-sensitive — pricing tiers, feature lists, and “best X” rankings change constantly, and a model’s training cutoff makes this the single most common failure mode in practice.
- Click through every citation the AI generated before it goes anywhere near publish. If a tool suggests a URL, confirm the URL resolves and the page actually says what’s claimed — never assume a plausible-looking source is real.
- Sanity-check quotes by searching the exact phrase. If it doesn’t turn up anywhere, treat it as a paraphrase and rewrite it as one, or cut it.
Tools that help — and their real limits
| Tool | What it actually checks | Real limitation |
|---|---|---|
| Originality.ai Fact Checker | Flags claims in a draft and surfaces a source to compare against, at 1 credit per 10 words of text scanned | Surfaces candidate sources — it doesn’t replace clicking through and reading them yourself |
| Perplexity Pro | Answers with inline citations for research and current-events queries | Independent testing has put its citation accuracy around 85–90% verifiable — meaning roughly 1 in 10 citations doesn’t fully support the claim it’s attached to |
| Direct vendor pages / primary sources | Ground truth for pricing, specs, and policy | Slower, but the only source that’s actually authoritative |
| A human subject-matter pass | Catches claims that are technically sourced but misleading in context | Doesn’t scale to bulk content the way the tools above do |
The realistic workflow is layered: run Originality.ai’s Fact Checker or ask Perplexity for a first pass on the claims worth verifying, then manually click through the sources it surfaces rather than trusting the summary. Both tools are a starting point for triage, not a substitute for reading the primary source.
A worked example
An AI draft on password managers once claimed a named competitor “supports biometric login on over 50 million devices.” That number wasn’t in the vendor’s own marketing, wasn’t attributed to any dataset, and didn’t turn up in a search of the vendor’s press releases — a clear invented-statistic pattern. The fix wasn’t to soften the number, it was to replace it entirely: the vendor’s actual claim, pulled from its support documentation, was that biometric login works on any device supporting the platform’s native biometric API (Face ID, Touch ID, Windows Hello), with no device-count figure published at all. The rewritten sentence was both more accurate and, honestly, more useful to a reader deciding whether their own device would work.
Red flags that should stop you before you publish
- A statistic with no attached source anywhere in the draft
- A quote you can’t find with an exact-phrase search
- Pricing, version numbers, or feature lists for any actively-updated product (these are wrong more often than not if the draft wasn’t checked against the vendor page the same week)
- A citation URL that returns a 404, redirects to an unrelated page, or leads to a domain that doesn’t match the claimed publisher
What not to trust AI to verify about itself
Don’t ask the same model that wrote the draft “is this accurate?” as your fact-check step — models are not reliably able to flag their own hallucinations, since the confidence of the output doesn’t correlate with its accuracy. A second, independent check — a different tool, a primary source, or a human — is the only thing that actually catches these errors.
The verdict
Fact-checking isn’t optional overhead on AI content — for open-ended generation specifically, it’s closer to a coin flip whether any given specific claim is exactly right. Budget real time for it: flag every checkable claim, verify against a primary source, click through every citation, and never let the model that wrote the draft be the one that approves it.
FAQ
How long should fact-checking take relative to writing?
For a claim-dense article (reviews, “best of” roundups, anything with pricing or stats), plan on 20–30 minutes of verification per 1,000 words — often close to as long as the drafting itself.
Are AI fact-checking tools like Originality.ai’s Fact Checker reliable on their own?
They’re useful for triage — flagging which claims need a look and surfacing a candidate source — but they don’t replace clicking through and confirming the source actually supports the claim.
Is Perplexity safe to use as a primary research source?
It’s strong for surfacing current, cited information quickly, but independent citation-accuracy testing puts it well short of 100%, so treat its citations as leads to verify, not finished sources to quote from directly.
What’s the single highest-risk type of claim to leave unchecked?
Anything time-sensitive — pricing, specs, or “current” rankings — because it’s the category most likely to be technically true when the model was trained and false by the time you publish.
