Why LLMs Hallucinate, and What Actually Reduces It

The uncomfortable truth first: a language model producing a fabricated citation is doing exactly the same thing as when it produces a correct one. There is no separate "making things up" mode that switches on. Understanding that is the difference between chasing the problem forever and actually reducing it.

The mechanism, without the metaphor

A language model estimates which token is likely to come next given everything before it. That is the whole operation. Training on an enormous amount of text makes those estimates very good, and good estimates of likely text correlate strongly with true text, because most text people wrote was trying to be true.

But the objective was never truth. It was plausibility. When the two diverge, the model follows plausibility, because that is the only thing it was ever optimised for.

Consider asking for a paper about a niche technique. The model has seen thousands of citations. It knows the shape: plausible author names, a title using the right vocabulary, a journal that publishes that sort of work, a year in the right range. Generating a citation that fits that shape is exactly the task it is good at. Whether that specific paper exists is a question the mechanism never asks.

This is why the word "hallucination" misleads. Nothing malfunctioned. The model interpolated in a region where it had pattern but no fact, which is what it does everywhere — usually to our benefit.

Why the model sounds so confident

There is a second, compounding cause, and it comes from the alignment stage rather than pretraining.

After pretraining, models are tuned on human preferences. Human raters, given a confident useful answer and a hedged uncertain one, reliably prefer the confident one. So the training signal rewards confidence. A model that said "I'm not sure" often would score worse, and so we trained that behaviour out.

The result is a system whose fluency is uniform and whose reliability is not. Correct answers and fabricated ones arrive in the same assured register. The absence of a hesitation signal is a trained-in property, not an oversight.

Where fabrication concentrates

Fabrication is not spread evenly, and knowing the hot spots tells you where to put your defences.

Specific identifiers are the worst: exact numbers, dates, citations, URLs, case references, version numbers, API method names. These are precisely the things that cannot be inferred from pattern, and precisely where a wrong answer looks most authoritative.

Rare entities. Anything the model saw a handful of times. It has enough to know the entity exists and not enough to say anything true about it — the worst possible combination.

Anything after the training cutoff. The model does not know what it does not know, so it answers from the world as it was.

False premises in your question. Ask "why does library X's parallel_map function block the event loop" and a model will frequently explain why, even if no such function exists. Your question asserted it, and agreeing is the more likely continuation.

Long chains of reasoning. Each step conditions on the last. One wrong step early gets confidently built upon.

Seven mitigations that measurably help

Ordered by how much they buy you.

1. Ground the answer in retrieved text. This is the big one and everything else is secondary. Retrieve the relevant documents and instruct the model to answer only from them. You are changing the task from "recall a fact" — which the architecture cannot guarantee — to "read this passage and summarise it", which it does reliably. If you build one defence, build this one.

2. Require citations to the provided context. Ask for the source of each claim, pointing into the supplied documents. This helps twice: it makes verification cheap for the user, and a claim with nothing to cite is much less likely to be produced in the first place.

3. Give the model an explicit way out. Add "if the context does not contain the answer, reply that you do not know" — and mean it, including examples of that being the right response. You are partly undoing the confidence training, and it works better than people expect.

4. Never state facts inside the question. Ask "does library X have a function for parallel mapping, and if so what is it called" rather than asserting it does. Removing the false premise removes the fabrication built on top of it.

5. Separate retrieval from reasoning. Rather than one prompt doing everything, have one step gather facts and a second reason over them. A short, well-scoped step has fewer places to drift.

6. Verify with a second pass. For high-stakes output, a second call that checks each claim against the source catches a meaningful share of errors, because checking is an easier task than generating. Better still, verify mechanically where you can: if the model emits a URL, fetch it; if it names an API method, check it against the real signature.

7. Lower the temperature — but do not expect much. Temperature near zero makes output more deterministic and slightly more conservative. It does not fix hallucination, because the most likely token can be confidently wrong. Useful, marginal, frequently oversold.

The thing you cannot skip

None of the above means anything if you cannot tell whether it helped.

Build a test set: fifty to a hundred real questions with verified answers, including questions your system should refuse. Run it after every prompt change, every model swap, every retrieval tweak. Track two numbers separately — how often it is right, and how often it is confidently wrong when it should have abstained. The second number is the one that damages user trust, and it is the one nobody measures.

Teams that skip this are not reducing hallucination. They are adjusting prompts and hoping, and they usually cannot tell you whether last month's changes made things better or worse.

Frequently asked questions

Will bigger models solve this?

They reduce it and do not remove it. Larger models know more, so they interpolate less often. But the objective is still plausibility, so the failure mode survives — it just moves to more obscure questions, which can make it harder to catch.

Does RAG eliminate hallucination?

It reduces it substantially and introduces its own failure: if retrieval returns the wrong passage, the model will faithfully summarise the wrong passage. You have traded fabrication for retrieval error, which is a good trade because retrieval error is measurable and fixable.

Can the model tell me how confident it is?

Asking it to rate its confidence produces a number that is itself generated text, and it is poorly calibrated. Token probabilities from the API are a somewhat better signal but still only loosely tied to factual accuracy. Treat self-reported confidence with suspicion.

Why does it invent function names in code?

Same mechanism, more visible. It has seen the library's naming conventions and generates a name that fits them. The saving grace is that code is checkable: run it, and the fabrication surfaces immediately. That is why AI-assisted coding works better than AI-assisted fact recall.

Is fine-tuning on my documents a fix?

No, and it is a common and expensive detour. Fine-tuning teaches the model the style of your documents without any guarantee that specific facts survive. Put knowledge in retrieval; use fine-tuning for behaviour. We covered the distinction in RAG vs fine-tuning.

Where to go deeper

The mitigations only make sense once the mechanism does. Our free Large Language Models course explains next-token prediction, attention and why context is the lever, and the free AI ethics and limitations course covers where these systems should not be trusted. For building systems that stay grounded, RAG Systems goes through retrieval quality, chunking and citation, which is where most of the real work lives.

The mental shift worth making: stop asking the model to remember, and start asking it to read.