AI & LLMs
Why LLMs hallucinate
Ask a model for a citation it's never seen and it may invent one — right format, plausible authors, journal that sounds real, and utterly fake. It's not lying; it's doing exactly what it was built to do. Understanding why is the key to trusting these systems in the right places.
It predicts, it doesn't retrieve
Under the hood, a language model does one thing: given the text so far, it estimates a probability for every possible next token and picks from the top. It has no database it "looks up" and no built-in sense of true vs false — only "what usually comes next." Most of the time, what usually comes next is the truth, because true statements are common in its training data. But when it doesn't actually know, the most probable continuation is whatever sounds right.
Why it sounds so sure
Fluency and accuracy are separate skills. The model is optimised to produce text that reads well — grammatical, confident, well-structured — regardless of whether the facts underneath are right. So a hallucination arrives in the same calm, authoritative tone as a correct answer. There's no built-in "confidence meter" surfaced to you, and the wrong answer often looks more polished than a hedged correct one.
The model always answers with the most likely-sounding words. When it doesn't know, "most likely-sounding" and "true" part ways — and you get a confident fabrication.
How to keep it honest
- Ground it in real data (RAG). Retrieve the actual documents and feed them in, so the most-probable answer is copied from a source rather than invented. This is where semantic search earns its keep.
- Give it tools. Let it call a database, calculator, or search instead of guessing — see tool use.
- Ask for citations you can verify, and treat any it can't back up as suspect.
- Lower the temperature for factual tasks so it favours the safest continuation over a creative one.
Hallucination isn't a bug that gets "fixed" — it's a direct consequence of how these models work. The winning move isn't to expect perfect recall; it's to design systems that give the model the facts, and to verify what matters.