Why "no fabricated sources" is a feature, not a slogan
Measured evidence: language models fabricate citations at scale, and fail nearly half the time when retrieving medical literature. Here is how Phở does it differently.
For a clinical decision, the dangerous thing is not an AI that says “I’m not sure”. The dangerous thing is an AI that answers fluently, confidently, and with a fabricated source. This is not a theoretical worry. It has been measured.
The problem, in real numbers
One study compared 5 free-tier LLM platforms on retrieving reference data for articles from BMJ, JAMA, and NEJM. The result: the platforms “completely failed to retrieve correct reference data 47.8% of the time” (Gao et al., 2026). Nearly half.
At larger scale, an audit of 111 million references estimated “a conservative estimate of 146,932 hallucinated citations in 2025 alone” (Zhao et al., 2026), rising sharply after widespread LLM adoption.
And the problem is fundamental: a benchmark of 10,923 prompts across nine domains found that “even the best-performing models are riddled with hallucinations (sometimes up to 86% of generated atomic facts depending on the domain)” (Ravichander et al., 2025 — HALoGEN).
Why models fabricate
Not because they are “careless”. An analysis from OpenAI argues that “language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty” (Kalai et al., 2025). Under binary right/wrong grading, “I don’t know” is always penalized, so models learn to guess confidently.
How Phở does it differently
If the problem is that models are rewarded for guessing, the product-level fix is to invert that reward:
- Ground on content in hand. Phở’s clinical mode answers from the real abstract it actually retrieved (PubMed/DOI), not from the model’s memory.
- Cite-or-abstain. A claim that isn’t backed by a source has its citation removed and is labelled “unverified” — never fabricated, never silently deleted.
- Measure and publish. We measure our own groundedness metric on our academic eval (~91%, measured on our task — a different metric than the per-model numbers above) and publish it, in that same honest spirit.
The 47.8% and 146,932 figures measure the literature and other platforms, not our product. We cite them to name the problem, not to flatter ourselves.
For a decision like switching antiplatelet therapy after a coronary intervention, “a source you can click to verify” is not a nice-to-have. It’s the minimum bar for trust. See a concrete case.