LLM Evaluation
Hallucination Detection and Mitigation
LLM hallucinations—plausible-sounding yet factually incorrect outputs—remain one of the most critical failure modes in production systems. This guide covers the full pipeline from detection to mitigation.
- Taxonomy — Intrinsic vs extrinsic, factual vs faithfulness hallucinations
- Detection — Reference-based, reference-free, and model-based approaches
- Mitigation — Retrieval augmentation, training-time fixes, and decoding strategies
The truth is rarely pure and never simple.
Hallucination Detection and Mitigation
LLM hallucinations—plausible-sounding yet factually incorrect outputs—remain one of the most critical failure modes in production systems. Understanding their taxonomy, detection, and mitigation is essential for building trustworthy AI applications.
Taxonomy of Hallucinations
Hallucinations can be classified along two axes: what is hallucinated and where the information should have come from.
| Type | Source of Error | Example |
|---|---|---|
| Intrinsic | Contradicts source | "The paper was published in 2021" when source says 2020 |
| Extrinsic | Fabricated detail | Inventing a citation that does not exist |
| Factual | World knowledge violation | "The Earth orbits the Sun in 400 days" |
| Faithfulness | Context ignored | Summarizing a document but adding unsupported claims |
The Hallucination Problem in Practice
Why Hallucinations Occur
Hallucinations arise from several fundamental properties of how LLMs are trained and how they generate text:
The fundamental tension is that LLMs are trained to predict what a human would write, not what is true. This fluency-truth gap is the root of the hallucination problem.
Impact Across Domains
| Domain | Hallucination Rate | Risk Level | Consequence |
|---|---|---|---|
| Medical Q&A | 15-25% | Critical | Misdiagnosis, wrong treatment |
| Legal advice | 10-20% | High | Invalid arguments, liability |
| Financial analysis | 8-15% | High | Incorrect projections, losses |
| Education | 5-12% | Medium | Misinformation propagation |
| Creative writing | 20-40% | Low | Acceptable (fiction) |
Detection Methods
Reference-Based Detection
When a ground-truth reference exists, we can compute factual overlap metrics:
Reference-Free Detection (Model-Based)
For open-ended generation where no reference exists, we use LLM-as-judge or trained classifiers:
The intuition: if a claim in y is factual, independent samples from the model should agree. If it is hallucinated, samples will diverge on the details.
NLI-Based Detection
Natural Language Inference models can check whether a source supports a generated claim:
Mitigation Strategies
Retrieval-Augmented Generation (RAG)
Grounding generation in retrieved evidence is the most effective mitigation:
Training-Time Mitigation
- Knowledge-grounded training: Train on (context, grounded-response) pairs
- Contrastive learning: Penalize hallucinated outputs, reward faithful ones
- RLHF with factuality reward: Reward model specifically scores factual accuracy
Decoding-Time Strategies
A key insight: reducing temperature and using smaller nucleus sampling regions reduces hallucination but also reduces creativity. This is the faithfulness-creativity tradeoff.
Evaluation Frameworks
| Framework | Method | Metric | Reference Required |
|---|---|---|---|
| FactScore | Atomic fact decomposition | Precision/Recall of facts | Optional |
| SelfCheckGPT | Multi-sample consistency | Agreement score | No |
| G-Eval | LLM-as-judge | Likert scale | Optional |
| TRU lens | Chain of verification | Claim verification rate | Yes |
| HaluEval | Hallucination detection QA | Binary classification | Yes |
Practice Exercises
-
Conceptual: Explain why SelfCheckGPT works for factual claims but may fail for subjective opinions. What properties of a claim make it amenable to consistency-based detection?
-
Mathematical: Given a hallucination rate of 5% per sentence and an average output of 10 sentences, compute the probability that at least one hallucination occurs in a generated response.
-
Practical: Implement a simple hallucination detector using an NLI model (e.g., DeBERTa-v3 fine-tuned on MNLI) that checks whether a source document entails a generated summary.
-
Research: Compare RAG-based mitigation with RLHF-based mitigation. Under what conditions does each approach dominate?
What to Learn Next
-> Bias and Fairness in LLMs Measuring and mitigating biases in language model outputs.
-> LLM Evaluation Frameworks Comprehensive evaluation methodologies for language models.
-> Automated LLM Evaluation Using models to evaluate models at scale.
-> Red Teaming Methodologies Systematic adversarial testing of language models.
-> RAG System Design Building retrieval-augmented generation systems for factual grounding.
-> LLM Benchmarking Suites Comprehensive benchmarks including hallucination evaluation.