The autocomplete analogy
Think about the autocomplete on your phone. As you type, it suggests the next word — not because it knows what you mean, but because it has seen which words tend to follow which. A large language model is the same idea at enormous scale: it predicts the next chunk of text based on patterns learned from huge amounts of writing. Most of the time that prediction is correct and true. But the model is optimised to produce text that sounds right, not text that is right — and when those two things come apart, you get a hallucination: a fluent, confident answer that is simply made up.
Why it sounds so sure
The unsettling part of hallucinations is the confidence. The model never says “I think” or “I’m not certain” unless it has learned that those words fit the pattern. A fabricated court case, a made-up statistic, and a real fact are all generated by exactly the same machinery — predicting plausible next words — so they all come out with the same polished, assured tone. There is no internal “truth meter” flashing red behind the scenes. To the model, a believable falsehood and a real fact are equally valid completions of the prompt.
When hallucinations are most likely
Hallucinations cluster around specific, verifiable details that live in the gaps of the training data. Ask for the general gist of a well-known topic and the model is usually reliable. Ask for an exact quote, a precise figure, a citation, a case number, an API parameter, or the biography of an obscure person, and the risk climbs sharply. The model would rather produce a plausible-looking answer than admit a gap, so it fills the hole with something that fits the shape of a real answer — a fake DOI, an invented author, a statistic that sounds about right.
How to protect yourself
You cannot fully eliminate hallucinations, but you can manage them. Give the model the source material so it summarises real text instead of recalling from memory — this is the whole point of retrieval-augmented generation. Ask it to cite where each claim comes from, which makes fabrications easier to spot. Tell it explicitly that “I don’t know” is an acceptable answer, which lowers the pressure to invent. And above all, verify anything specific — names, numbers, quotes, links — against a trusted source before you rely on it.
The right mental model
Treat the AI as a fast, fluent, well-read assistant with no fact-checker and no shame about gaps. Its output is a confident first draft, not a verified reference. Used that way — for drafting, brainstorming, summarising material you provide, and explaining concepts — it is enormously useful. Used as an oracle of precise facts, it will eventually hand you a beautifully written falsehood with total conviction. Knowing why that happens is the single best defence against it.