On this page
Read
The useful mental model
A language model is built to continue with plausible language. If the available evidence stops before the answer is complete, fluent generation can continue past the evidence boundary unless the surrounding system checks support or allows the model to abstain.
In plain English
The AI sounds sure, but it made things up.
Where builders use it
- 01Risk reviews in legal/finance
- 02Fact-checking pipelines
- 03Human approval workflows
Why it matters
Fluent wording can make an unsupported claim look trustworthy. In consequential workflows, that can turn a generation error into a customer, compliance, or operational failure.
Why now
Newer models reduce some hallucinations, but current primary research and risk guidance still treat confident false output as an unresolved problem that requires evaluation, grounding, abstention, and human review where stakes are high.
A common misconception
“Adding RAG or citations eliminates hallucinations.”
Retrieval and citations can reduce unsupported output, but retrieval can miss the right evidence and generated citations can themselves be wrong. The workflow must verify that the cited source actually supports the claim.
See
See the mental model
The visual is generated from the same structured knowledge as the written explanation.
A customer question reaches an approved policy that does not contain the requested exception. Instead of continuing into an unsupported claim, the guarded workflow verifies support and abstains or escalates.
Watch
Watch the explainer
A short visual explanation built from the same reviewed knowledge object.
Hallucination: fluency can continue after evidence ends
See how a support assistant crosses beyond an approved policy, then checks the unsupported claim, abstains, and routes the case for human review.
Read transcript
A customer asks an assistant for a special refund exception. The approved policy covers only standard refunds. A language model keeps writing plausible text, even when the evidence runs out. So it may invent a rule that sounds helpful and confident. That is a hallucination, fluent text that goes beyond what the evidence supports. Giving the model the policy helps, but it proves nothing. Even a citation can be wrong. A guarded workflow checks each important claim against the real passage. Here, nothing in the policy supports the exception. So the assistant says it is unable to confirm, and sends the case to a person. A polished sentence is no substitute for evidence.
Builder example
Stop an unsupported refund exception
A customer asks a support assistant whether a workshop booking can receive a special refund. The approved policy explains standard refunds but contains no special exception for this case.
Failure
The model completes a helpful-sounding answer by inventing an exception that is not present in the approved policy.
Intervention
Retrieve the relevant policy, require every consequential claim to match a supporting passage, and instruct the workflow to abstain or escalate when the evidence is missing.
Observable result
The customer receives either a source-supported policy answer or a clear handoff for human review, rather than a fabricated exception.
Sources
Go deeper
Last researched 11 October 2026