LLM Hallucination Detection in Production: Semantic Entropy, Self-Consistency, NLI Grounding Checks and Guardrails Compared (2026)
Most teams that ship a “hallucination filter” are running one detector against two different failure modes and wondering why it misses half the bad answers. A model can invent a fact that no source contains. It can also contradict the very document you handed it. Those are different problems, they leave different traces, and they need different detectors. LLM hallucination detection in 2026 is no longer a research curiosity. Vendor guardrails ship grounding scores and a Nature paper gave sampling-based uncertainty a rigorous footing, yet detectors still get chosen by popularity rather than by the error they can see. This explainer maps each major detector to the error it catches. It shows working Python for two of them, measures them correctly, and wires them into a router that serves, hedges or abstains.
What this covers: a working taxonomy of hallucinations, how semantic entropy, SelfCheckGPT, self-consistency, token log-probabilities and NLI grounding checks work, runnable code, a cost and latency table, evaluation with AUROC and precision at fixed recall, a production architecture, failure modes and a decision table.
Context and Background
The word “hallucination” covers too much ground to be useful on its own, so the literature splits it. Ji et al.’s survey in ACM Computing Surveys (2022) popularised the split between intrinsic hallucinations and extrinsic ones. An intrinsic hallucination contradicts the source the model was given. An extrinsic one adds content the source cannot verify, either true or false. A second axis matters just as much in practice. In closed-domain tasks, such as summarisation, retrieval-augmented generation (RAG) and extraction, there is a reference text to check against. In open-domain tasks, such as “who founded this company?” with no context, the only reference is the world, and the model’s own parameters are the only thing you can interrogate cheaply.
Detection research grew along those two axes. For closed-domain faithfulness, the dominant tool is natural language inference (NLI): a classifier that decides whether a premise entails, contradicts or is neutral toward a hypothesis. Purpose-built descendants include Vectara’s Hughes Hallucination Evaluation Model (HHEM), MiniCheck from Tang, Laban and Durrett, and the LLM-judged faithfulness metric in Ragas. For open-domain factuality, the dominant idea is consistency under resampling. SelfCheckGPT (Manakul, Liusie and Gales, EMNLP 2023) samples several answers and checks whether they agree. Semantic entropy (Farquhar, Kossen, Kuhn and Gal, Nature, 2024) formalises this as entropy over meanings rather than over strings.
A third strand explains why hallucination is so persistent. In “Why Language Models Hallucinate” (2025), Kalai, Nachum, Vempala and Zhang argue that pretraining makes some errors statistically inevitable. They add that binary right-or-wrong benchmark scoring then rewards guessing over abstaining. It predicts the errors you will see most: confident guesses on rare facts.
This post sits next to two existing pieces on the site. The span-level citation and attribution guide covers how to make RAG answers cite evidence. The fact-check on viral AI hallucination videos covers the public-facing myths about LLM uncertainty. Here the focus is operational: which detector, at what cost, measured how, wired where. The primary source for the uncertainty side is the semantic entropy paper in Nature.
How LLM Hallucination Detection Works: Two Problems, Five Signal Families
LLM hallucination detection is the task of scoring a generated answer, or each claim inside it, for the probability that it is unsupported or false. Practical detectors read one of five signals: agreement with supplied evidence (NLI grounding), agreement across resampled answers (self-consistency, SelfCheckGPT, semantic entropy), token probabilities, internal hidden states, or a judge model’s verdict.

Figure 1: A production LLM hallucination detection pipeline. Grounding and uncertainty are scored on separate paths and merged in a policy router.
Figure 1 shows the reference architecture this post builds toward. The generator’s answer takes two paths. The upper path splits the answer into atomic claims and checks each against retrieved evidence with an NLI model; this catches intrinsic and extrinsic errors relative to the context. The lower path draws extra samples and scores the spread of their meanings; this catches confabulation where no context exists or the context is silent. A policy router combines both scores with route-specific thresholds. It then serves the answer, hedges by marking unsupported spans, or abstains and escalates. Every decision lands in a trace store that feeds the labelled evaluation set.
Grounding errors are a classification problem you can supervise
When a source document exists, “is this claim supported by this text?” is a well-posed classification question. It has labelled datasets, mature architectures and a clear decision boundary. That is why small models do surprisingly well here. HHEM-2.1-Open is a fine-tuned FLAN-T5-base of roughly 0.1 billion parameters, released under Apache-2.0. Vectara’s model card says it runs in under 600 MB of RAM at 32-bit precision and takes about 1.5 seconds for a 2,000-token input on a modern x86 CPU. MiniCheck’s authors report that their 770M-parameter MiniCheck-FT5 reaches GPT-4-level accuracy on their LLM-AggreFact benchmark at roughly 400 times lower cost.
Grounding checks are cheap because the evidence carries most of the information. The checker does not need to know the world. It only needs to read two passages and compare them. That also defines the limit. A grounding check cannot tell you whether the evidence is right, and it cannot help when retrieval returned nothing relevant.
Uncertainty errors need the model to argue with itself
Without a reference text, the only cheap evidence is the model’s own behaviour. The core empirical observation, stated plainly in the SelfCheckGPT abstract, is this. When a model knows a fact, independently sampled answers tend to agree. When it is guessing, samples diverge and contradict each other. That observation underlies several detectors.
Self-consistency (Wang et al., ICLR 2023) was introduced as a decoding method, not a detector. It samples diverse chain-of-thought reasoning paths and takes the majority final answer. The paper reports gains over greedy chain-of-thought of +17.9% on GSM8K and +11.0% on SVAMP, among other benchmarks. The agreement rate of the majority vote is a natural confidence score as a by-product. It works only when answers have a canonical form, such as a number or a multiple-choice letter, that you can compare by string equality.
SelfCheckGPT generalises agreement to free text. It samples several stochastic responses, then scores each sentence of the main response by how well the samples support it. The paper offers several scorers: BERTScore, question answering, n-gram, NLI and LLM prompting. It evaluated them on GPT-3 biographies of WikiBio subjects with manual factuality labels. Its key property is that it is black-box and zero-resource: no logits, no database.
Semantic entropy tightens the idea. Two answers like “Paris” and “It’s Paris, France” are different strings with the same meaning. Naive entropy over strings would call that uncertainty. Semantic entropy clusters samples by mutual entailment first, then computes entropy over clusters. High entropy means the model is spreading probability across different meanings. The Nature authors call that pattern confabulation: arbitrary, incorrect generations.
Log-probabilities and hidden states are cheap but leaky
If you control the model or the API exposes token log-probabilities, you get a signal at no extra generation cost. Common scores are mean negative log-probability across the answer, the maximum token surprisal, and per-token predictive entropy. These were the classic “grey-box” baselines that SelfCheckGPT compared against and reported beating. A related idea asks the model to grade itself: Kadavath et al. (2022) showed that larger models can estimate P(True) for their own proposed answers with useful calibration when the question format is right. The weakness is that token probability measures lexical uncertainty, not semantic uncertainty. A model can be unsure whether to write “the 1990s” or “1990s” while being certain of the decade. Instruction tuning and RLHF also tend to sharpen token distributions, which weakens raw probability as a calibration signal.
Judge-style self-verification is a fifth option. Chain-of-Verification (Dhuliawala et al., 2023) drafts an answer, plans verification questions, answers them independently and then revises. It is closer to mitigation than detection, but its verification answers double as a cheap disagreement signal.
Hidden-state probes are the newest family for LLM hallucination detection. Kossen et al.’s “Semantic Entropy Probes” (2024) train a linear probe on the hidden states of a single generation to predict semantic entropy. The paper notes that full semantic entropy multiplies compute by roughly 5 to 10 times. It claims probes cut the overhead to “almost zero” and generalise out of distribution better than probes trained to predict accuracy directly. The catch is operational: you need white-box access to activations, so probes are an option only for self-hosted open-weight models.
Deeper Analysis: Semantic Entropy and NLI Grounding in Code
This section walks through the two detectors most teams should understand end to end. One handles uncertainty, one handles grounding. Together they cover the two halves of Figure 1.
Semantic entropy, step by step

Figure 2: Semantic entropy. Samples are grouped into meaning clusters by bidirectional entailment, then entropy is computed over clusters rather than strings.
Figure 2 traces the algorithm described in Kuhn, Gal and Farquhar’s ICLR 2023 paper and extended in the 2024 Nature paper. First, sample N answers to the same prompt at a non-zero temperature. Second, compare answers pairwise with an NLI model in both directions. Two answers share a meaning only if A entails B and B entails A, which is the “bidirectional entailment” test. Third, merge answers into clusters. Fourth, give each cluster a weight. With token probabilities, the weight is the summed sequence probability of its members. Without them, the weight is simply the fraction of samples in the cluster, which the Nature paper calls discrete semantic entropy. Finally, compute Shannon entropy over the cluster weights.
The original ICLR paper used a DeBERTa-large model fine-tuned on MNLI as the entailment judge. It drew 10 samples per question and reported, for OPT-30B, AUROC of 0.83 on TriviaQA and 0.77 on CoQA. That beat simply counting distinct answers, which scored 0.79 and 0.66. The authors’ companion blog post says “5 or so” samples is often enough. It estimates the full method costs “in the ballpark of 10x” the compute of a plain answer.
Here is a compact, runnable implementation of discrete semantic entropy. It uses an open NLI cross-encoder from the sentence-transformers ecosystem. Its model card documents the label order as contradiction, entailment, neutral.
import math
from sentence_transformers import CrossEncoder
NLI = CrossEncoder("cross-encoder/nli-deberta-v3-base")
LABELS = ["contradiction", "entailment", "neutral"] # order per model card
def _with_q(question: str, answer: str) -> str:
# Prepend the question so terse answers like "1879" carry context.
return f"Q: {question} A: {answer}"
def bidirectional_entails(question: str, a: str, b: str) -> bool:
pa, pb = _with_q(question, a), _with_q(question, b)
probs = NLI.predict([(pa, pb), (pb, pa)], apply_softmax=True)
return all(LABELS[row.argmax()] == "entailment" for row in probs)
def cluster_by_meaning(question: str, answers: list[str]) -> list[list[str]]:
clusters: list[list[str]] = []
for ans in answers:
for cl in clusters:
# Compare against the cluster representative only (greedy).
if bidirectional_entails(question, cl[0], ans):
cl.append(ans)
break
else:
clusters.append([ans])
return clusters
def discrete_semantic_entropy(question: str, answers: list[str]):
clusters = cluster_by_meaning(question, answers)
n = len(answers)
weights = [len(c) / n for c in clusters]
entropy = -sum(w * math.log(w) for w in weights)
return entropy, clusters
def semantic_entropy_with_logprobs(question, answers, seq_logprobs):
"""seq_logprobs: length-normalised log p(answer | prompt) per sample."""
clusters = cluster_by_meaning(question, answers)
idx = {a: i for i, a in enumerate(answers)}
cluster_lp = []
for c in clusters:
lps = [seq_logprobs[idx[a]] for a in c]
m = max(lps)
cluster_lp.append(m + math.log(sum(math.exp(x - m) for x in lps)))
# Renormalise over observed clusters, then take entropy.
m = max(cluster_lp)
z = m + math.log(sum(math.exp(x - m) for x in cluster_lp))
p = [math.exp(x - z) for x in cluster_lp]
return -sum(pi * math.log(pi) for pi in p if pi > 0)
if __name__ == "__main__":
q = "In which year was the Eiffel Tower completed?"
samples = ["1889", "It was completed in 1889.", "1889",
"1887", "In 1889.", "1889", "1888", "1889"]
h, cl = discrete_semantic_entropy(q, samples)
print(round(h, 3), [len(c) for c in cl])
A few design choices in this code deserve justification. The greedy loop compares each new answer only with a cluster’s first member. That keeps NLI calls close to linear in N when most answers agree, rather than the full N(N-1) directed pairs. It is the same “not too far off linear for small K” behaviour the authors describe. It also means clustering is order-dependent when entailment is not transitive, which it often is not. Prepending the question is a deliberate choice for short-answer QA: an NLI model given only “1887” and “1889” has little to work with. And apply_softmax=True matters because cross-encoders return raw logits by default. Check the flag against your installed sentence-transformers version.
Now a worked example, labelled illustrative. Suppose 8 samples split into clusters of sizes 5, 2 and 1. Discrete semantic entropy is −(0.625 ln 0.625 + 0.25 ln 0.25 + 0.125 ln 0.125) ≈ 0.294 + 0.347 + 0.260 ≈ 0.90 nats. If all 8 agree, entropy is 0. If all 8 differ, it is ln 8 ≈ 2.08 nats. The useful range for thresholding therefore sits between 0 and ln N. That means thresholds do not transfer across different N. Recalibrate whenever you change the sample count.
Scaling semantic entropy to paragraphs
Short-answer QA is where semantic entropy was validated most heavily, but production answers are paragraphs. The Oxford group’s write-up describes the extension. Decompose the long answer into specific claims. For each claim, generate questions that could have produced it. Sample answers to those questions and compute semantic entropy per claim. This is effective but expensive. A 6-claim answer with 3 questions per claim and 5 samples each costs 90 extra short generations before NLI clustering. That number is illustrative arithmetic, not a benchmark. Budget it like a batch job, not a request-path check, unless the route is high stakes and low volume.
NLI grounding checks, claim by claim

Figure 3: A claim-level grounding check. Each atomic claim is paired with evidence windows and scored for entailment and contradiction.
Figure 3 shows the grounding path. The answer is split into atomic claims. Each claim is paired with candidate evidence: the chunks it cites, plus the top-k retrieved chunks as a fallback. An NLI model scores every pair. The policy takes, per claim, the maximum entailment probability across evidence and the maximum contradiction probability. A claim is supported if some chunk entails it. It is contradicted if some chunk contradicts it strongly and none entails it. Otherwise it is unsupported.
That three-way outcome matters more than the score. “Contradicted” is an intrinsic hallucination and should almost always block or rewrite. “Unsupported” is extrinsic. It may be true world knowledge, or it may be invention, and the right action depends on the route.
import re
import numpy as np
from sentence_transformers import CrossEncoder
NLI = CrossEncoder("cross-encoder/nli-deberta-v3-base")
C, E, N = 0, 1, 2 # contradiction, entailment, neutral
def split_claims(answer: str) -> list[str]:
# Naive sentence split; swap in an LLM claim decomposer for long answers.
parts = re.split(r"(?<=[.!?])\s+", answer.strip())
return [p for p in parts if len(p.split()) >= 4]
def windows(chunk: str, size: int = 120, stride: int = 60) -> list[str]:
# Keep premises well under the cross-encoder's input limit.
words = chunk.split()
if len(words) <= size:
return [chunk]
return [" ".join(words[i:i + size])
for i in range(0, len(words) - size + stride, stride)]
def grounding_report(answer: str, chunks: list[str],
entail_t: float = 0.5, contra_t: float = 0.7):
evidence = [w for ch in chunks for w in windows(ch)]
report = []
for claim in split_claims(answer):
probs = NLI.predict([(ev, claim) for ev in evidence],
apply_softmax=True)
best_e = float(probs[:, E].max())
best_c = float(probs[:, C].max())
src = int(probs[:, E].argmax())
if best_e >= entail_t:
label = "supported"
elif best_c >= contra_t:
label = "contradicted"
else:
label = "unsupported"
report.append({"claim": claim, "label": label,
"entail": round(best_e, 3),
"contradict": round(best_c, 3),
"evidence": evidence[src][:160]})
supported = sum(r["label"] == "supported" for r in report)
score = supported / max(len(report), 1)
return score, report
if __name__ == "__main__":
ctx = ["Albert Einstein was born in Ulm, in the Kingdom of Wurttemberg "
"in the German Empire, on 14 March 1879."]
ans = ("Einstein was born in Germany. "
"Einstein was born on 20 March 1879. "
"He later won the Nobel Prize in Physics.")
score, rep = grounding_report(ans, ctx)
print(score)
for r in rep:
print(r["label"], r["entail"], r["contradict"], "|", r["claim"])
The aggregate score, supported claims over total claims, is the same shape as the Ragas faithfulness metric. Ragas documents it as the number of claims supported by the retrieved context divided by the total number of claims. Its documentation uses the same Einstein example, where a wrong birth date yields a score of 0.5. Ragas uses an LLM for the support judgment. It also offers a FaithfulnesswithHHEM variant that swaps in HHEM-2.1-Open as a cheaper classifier. In the example above, a well-trained NLI model should mark the first claim supported and the second contradicted. It should mark the Nobel claim unsupported, because it is true but absent from the context. The exact probabilities depend on the checkpoint, so run the script rather than trusting predicted output.
Three implementation details separate a toy from a usable checker. Windowing: general NLI cross-encoders are trained on sentence-length pairs and have bounded input length. Long premises are either truncated, which silently drops evidence, or diluted. HHEM-2.x and MiniCheck were built specifically for document-length premises, which is one reason to prefer them over a generic NLI checkpoint. Max-pooling across evidence: a claim needs only one supporting window, so take the max, not the mean. Claim decomposition: naive sentence splitting fails on compound sentences such as “X was founded in 2015 and acquired by Y.” Half of that sentence can be right and half wrong. FActScore (Min et al., EMNLP 2023) formalised the alternative. It splits text into atomic facts and computes the share supported by a knowledge source. The paper reports ChatGPT biographies scoring 58%, with an automated estimator below 2% error relative to human judgments.
Cost and latency per detector
The table below compares the detector families on the resources they consume. All numbers are illustrative order-of-magnitude estimates for a typical RAG answer of about 150 output tokens, 3,000 tokens of context and 6 claims. They are not benchmarks. Measure on your own stack.
| Detector | Signal | Access needed | Extra LLM calls | Extra latency, illustrative | Catches best |
|---|---|---|---|---|---|
| Token log-prob or entropy | Lexical uncertainty | Logits from the serving API or model | 0 | Near zero | Low-confidence spans, if calibrated |
| Hidden-state probe | Learned uncertainty | White-box activations | 0 | Near zero | Confabulation, open-weight models only |
| NLI grounding check, small model | Evidence entailment | Retrieved context | 0, small classifier | Tens to hundreds of ms on GPU | Intrinsic and extrinsic RAG errors |
| LLM-judge faithfulness | Evidence entailment | Retrieved context | 1 or more judge calls | Seconds | Nuanced paraphrase, multi-hop support |
| Self-consistency vote | Answer agreement | Sampling | N minus 1 | Parallel: about 1 generation, plus tail | Short canonical answers |
| SelfCheckGPT | Sentence-sample agreement | Sampling | N minus 1, plus scorer | 1 to several generations | Open-domain long-form text |
| Semantic entropy, short answer | Meaning-cluster entropy | Sampling plus NLI | N minus 1, plus NLI | Parallel: about 1 generation, plus NLI | Confabulation on factoid questions |
| Semantic entropy, long form | Per-claim entropy | Sampling plus NLI | Tens of short calls | Batch scale | Confabulation inside paragraphs |
Two patterns stand out for LLM hallucination detection budgets. First, sampling-based detectors cost tokens in proportion to N, but parallel sampling keeps latency near one generation, plus a tail penalty from waiting on the slowest stream. Second, the cheapest detectors need either context (grounding) or white-box access (probes, logits). A team on a closed API with no retrieval has only the sampling family available, at the highest token cost. Vectara’s HHEM-2.1 announcement gives one real reference point. It says a GPT-4-backed Ragas faithfulness judgment can take up to 35 seconds on a 4,096-token context, against 0.6 seconds for HHEM-2.1 on an RTX 3090.
Hallucination Rate Measurement: Evaluating the Detectors Themselves
A detector is a binary classifier with a threshold, so evaluate it like one. Most teams skip this step of LLM hallucination detection. They pick a detector because a leaderboard ranked it, set the threshold to 0.5 and ship. That is how you end up blocking 15% of good answers while catching only half the bad ones.

Figure 4: The evaluation loop. Detectors are scored on labelled production traces, thresholds are set per route, and enforcement starts in shadow mode.
Figure 4 is the loop. Sample real traces, stratified by route and by retrieval quality. Label them at the claim level, because a single wrong number in a correct paragraph is the common case. Run every candidate detector offline on the same set. Choose thresholds per route from the precision-recall curve, not from a default. Run in shadow mode, flagging but still serving, for at least one traffic cycle. Then enforce with an explicit abstention budget. Keep monitoring flag rate and the detector’s score distribution for drift.
AUROC is necessary, not sufficient
Area under the ROC curve (AUROC) measures how well a score ranks hallucinated answers above correct ones, independent of threshold. It is the headline metric in the semantic entropy papers, and it is the right metric for comparing detectors in research. The Nature paper also reports AURAC, the area under the rejection-accuracy curve. It answers the question operators care about: if I refuse the most uncertain X% of questions, how accurate is what remains?
In production, the base rate dominates everything. Here is a worked example, labelled illustrative. Take 2,000 labelled answers with an 8% hallucination rate, so 160 positives and 1,840 negatives. A detector set to 80% recall flags 128 true hallucinations. If its false-positive rate at that threshold is 10%, it also flags 184 good answers. Precision is 128 / 312 ≈ 41%. Most flagged answers are fine. If the route’s base rate drops to 2%, the same detector at the same threshold gives 32 true flags against 196 false ones, for about 14% precision. A detector that “works” on a 50/50 research benchmark can be operationally useless on a well-grounded route. That is why AUPRC and precision at fixed recall belong in the scorecard next to AUROC.
Label the right unit, with the right reference
Three labelling decisions change your numbers more than the choice of detector.
The first is unit. Answer-level labels hide partial errors. Claim-level labels, in the FActScore style, expose them but cost more to annotate.
The second is reference. Decide whether “correct” means faithful to the provided context or true in the world. A RAG answer that faithfully repeats an outdated document is faithful but false. A grounding check will pass it, and that is the correct behaviour for a grounding check. Write the rubric down before labelling.
The third is source of positives. Public benchmarks are useful for sanity checks. RAGTruth provides about 18,000 RAG responses with case-level and word-level annotations. TruthfulQA has 817 questions in 38 categories designed to elicit imitative falsehoods, where the original best model was truthful on 58% of questions against 94% for humans. But their error distribution is not yours. A support bot’s hallucinations cluster around product SKUs, prices and policy dates, not trivia. If you use an LLM to pre-label, treat it as a detector with its own error rate. The pitfalls of that approach are covered in the site’s guide to LLM-as-a-judge bias and calibration.
Leaderboard hallucination rates are not your hallucination rate
Vectara’s hallucination leaderboard is the best-known public measurement. It asks models to summarise documents and scores the summaries with HHEM. In November 2025 Vectara relaunched it. The new dataset grew from 1,000 to over 7,700 articles of up to 32,000 tokens, across domains such as law, medicine and finance. A summary scoring below 0.5 counts as a hallucination. Vectara reported that several frontier reasoning models had rates above 10% on the new set, and that rates were generally higher than on the original benchmark. Vectara’s own researchers have also argued, in “Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards” (Tamber et al., 2025), that existing detectors have real shortcomings. They proposed FaithJudge, an LLM-judge approach anchored on human-annotated examples.
Leaderboard numbers measure summarisation faithfulness under one prompt, scored by one detector. They help model selection for summarisation-like routes and say nothing about your retrieval, prompt or open-domain factuality.
Why abstention has to be scored
Kalai et al.’s argument has a direct measurement consequence. If your internal evals score only accuracy, every model and prompt change that increases guessing will look like an improvement. OpenAI’s accompanying post illustrates this with SimpleQA figures from the GPT-5 system card. gpt-5-thinking-mini abstained on 52% of questions, was accurate on 22% and wrong on 26%. o4-mini abstained on 1%, was accurate on 24% and wrong on 75%. By accuracy alone, the second model wins. By error rate, it is roughly three times worse. The post’s recommendation, “penalize confident errors more than you penalize uncertainty”, applies equally to scoring your detectors and your router. Report three numbers per route: answered-and-correct, answered-and-wrong, and abstained.
Guardrails in Practice: What Vendors Actually Ship
Managed guardrails package the same mechanisms with operational defaults. Knowing which mechanism sits underneath tells you which errors they can see.
Amazon Bedrock Guardrails’ contextual grounding check is a grounding detector. Its documentation describes two scores. Grounding asks whether the response is supported by the supplied source. Relevance asks whether it answers the query. Thresholds run from 0 to 0.99, and responses below threshold are filtered. The documented limits are 100,000 characters of grounding source, 1,000 characters of query and 5,000 characters of response to guard. The docs note that with streaming, an irrelevant response can reach the user before it is flagged. They also describe two tiers. The Classic tier treats even reasonable inferences as ungrounded and does not support conversational use. The Standard tier tolerates reasoning and paraphrase, with 0.7 suggested as a starting threshold for conversational use. That is a vendor recommendation, not a universal constant.
NVIDIA NeMo Guardrails exposes both families as configurable rails. self check facts is an LLM-prompted grounding check against retrieved chunks; responses scoring below 0.5 are refused. self check hallucination is a SelfCheckGPT-style consistency check that samples extra responses, two by default, and needs no retrieval. The catalogue also integrates AlignScore and Patronus Lynx as model-based grounding checkers.
The practical takeaway: a managed grounding check is an NLI-family detector with a tuned model and a threshold. It inherits every limit of grounding checks, discussed below. A managed self-check rail is a sampling-family detector, with a sample count far lower than the 5 to 10 the semantic entropy authors suggest. Treat both defaults as starting points to calibrate on your labelled set.
Trade-offs, Gotchas, and What Goes Wrong
Consistent errors are invisible to consistency detectors. Every sampling method assumes that guessing produces diverse answers. If a model learned a wrong fact firmly, all ten samples agree and entropy is zero. The Oxford write-up says plainly that semantic entropy will not catch models trained into incorrect facts or reasoning styles. These systematic errors are often the most damaging, because they repeat across users.
Grounding checks trust the evidence. An NLI check certifies consistency with retrieved text, not truth. Stale documents, a wrong chunk ranked first, or a prompt-injected document all pass a faithful answer straight through. Grounding-based LLM hallucination detection is therefore only as good as your retrieval evaluation.
Entailment is not transitive, and paraphrase is not always equivalence. Greedy clustering can merge “the brain” with “the central nervous system” in one context and split “1889” from “March 1889” in another. Both examples come from the authors’ own failure discussion. Units, dates, negations and numeric ranges are where NLI models are weakest, and they are exactly where business answers carry risk.
Temperature and N are coupled to the threshold. Sampling at 0.3 makes everything look certain, while sampling at 1.2 makes everything look uncertain. Change either parameter and your calibrated threshold is wrong. The site’s explainer on temperature, top-p and min-p sampling covers why the distribution shifts.
Claim splitting is a hidden model. If an LLM decomposes answers into claims, its errors become your detector’s errors. It may drop a hedge (“reportedly”), merge two facts, or invent a claim the answer never made. Version and evaluate the decomposer like any other model.
Streaming defeats post-hoc checks. A check that runs on the full response cannot un-send tokens already displayed. Either buffer high-risk routes, check per sentence as it completes, or accept that the check becomes a retraction mechanism rather than a gate.
Over-blocking has a cost too. A threshold that refuses many answers trains users to rephrase until the filter passes, or to leave. Track abstention rate as a product metric with an owner.
Detector drift is silent. When you upgrade the generator, its fluency and hedging style change. A grounding model tuned on the old model’s outputs can shift its score distribution with no code change. Re-run the offline evaluation on every generator, prompt or retriever release. Gate the release on it, as described in the site’s guide to prompt and model version management with eval gates.
Decision Table: Which Detector for Which Route
The right LLM hallucination detection setup follows from two questions: is there evidence to check against, and what model access do you have? The table maps common routes to a primary and a secondary detector. It reflects the mechanisms described above and is the author’s recommendation, not a measured ranking.
| Route | Evidence available | Primary detector | Secondary detector | Default action on flag |
|---|---|---|---|---|
| RAG Q&A over internal docs | Yes | Claim-level NLI grounding, HHEM or MiniCheck class | LLM-judge faithfulness on a sampled audit | Hedge unsupported claims, block contradicted |
| Summarisation of a supplied document | Yes, complete | NLI grounding | Spot-check with FaithJudge-style judge | Rewrite or block |
| Open-domain factoid Q&A, no retrieval | No | Discrete semantic entropy, N of 5 to 10 | Log-prob entropy if exposed | Abstain or route to retrieval |
| Math or structured answers | No | Self-consistency vote agreement | Tool or solver verification | Abstain below agreement threshold |
| Long-form open-domain writing | Partial | Per-claim retrieval plus NLI, FActScore style | SelfCheckGPT on flagged sentences | Mark unverified claims |
| Self-hosted open-weight model, high volume | Either | Semantic entropy probe on hidden states | NLI grounding where context exists | Escalate high-score cases to sampling check |
| Regulated or high-stakes answers | Yes | NLI grounding plus semantic entropy | Human review queue | Abstain and escalate |
Note two things. First, the high-volume row uses a cascade: a near-zero-cost probe or log-prob score filters most traffic, and only the uncertain tail pays for sampling. This mirrors model cascades for cost-quality trade-offs. Second, no row relies on a single detector for regulated content. Grounding and uncertainty see different errors, and the systematic-error blind spot means human review stays in the loop.
Practical Recommendations
Start by classifying your routes, not by choosing a tool. For each route, write down whether evidence exists and what “correct” means: faithful to context, or true in the world. Those two answers pick the detector family for you. Then build the labelled set before you build the filter. Two or three hundred claim-level labels per high-traffic route, drawn from real traces, will tell you more than any public leaderboard. Wire detectors into your tracing so every score, threshold and decision is attached to the request. The site’s LLM observability architecture guide covers the plumbing.
For request-path LLM hallucination detection, prefer small, purpose-built grounding classifiers, and reserve LLM judges for offline audits and ambiguous cases. Use semantic entropy where there is no context to check against, and budget it as a multiple of generation cost. Treat abstention as a first-class outcome with a user experience. “I could not verify this; here is what the sources do say” is better than a refusal and much better than a fluent guess. If you need statistical guarantees on the error rate of answered questions, look at conformal prediction to turn detector scores into coverage-controlled thresholds.
A short checklist:
- Split routes by evidence availability, and record the correctness reference for each.
- Label 200 or more real traces per major route at the claim level.
- Evaluate detectors with AUROC, AUPRC and precision at your target recall, at your real base rate.
- Set thresholds per route; recalibrate whenever N, temperature, generator or retriever change.
- Distinguish contradicted claims from unsupported ones in the action policy.
- Run in shadow mode first, then enforce with an explicit abstention budget.
- Report correct, wrong and abstained rates together on every release.
Frequently Asked Questions
What is the most accurate LLM hallucination detection method?
There is no single most accurate method, because detectors see different errors. When a source document exists, claim-level NLI grounding checks with purpose-built models such as HHEM or MiniCheck are the most cost-effective. MiniCheck’s authors report GPT-4-level accuracy on their benchmark at far lower cost. Without context, semantic entropy is the best-validated black-box method for confabulations. Its Nature paper shows it beating naive entropy and other baselines across datasets and models. Neither catches errors the model makes consistently, so high-stakes routes combine both with human review.
How does semantic entropy detect hallucinations?
Semantic entropy samples several answers to the same prompt and groups them by meaning. Two answers share a cluster when an NLI model judges that each entails the other. It then computes entropy over the clusters instead of over raw strings, so paraphrases do not count as disagreement. If most samples land in one cluster, the model is consistent and entropy is low. If samples spread across contradictory meanings, entropy is high, signalling a likely confabulation. The original work used about 10 samples. The authors note that around 5 is often enough.
What is the difference between SelfCheckGPT and semantic entropy?
Both rely on the same observation: models that know a fact answer consistently across samples. SelfCheckGPT scores each sentence of a main response by how well a set of sampled responses supports it. It offers several scorers, including NLI, question answering, n-gram and LLM prompting, and targets long passages. Semantic entropy scores the spread of meanings across sampled answers to one question, using bidirectional entailment clustering. It is better suited to short answers. Both are black-box. Semantic entropy has the more formal probabilistic grounding and the larger published evaluation.
Can an NLI grounding check detect hallucinations without retrieval?
No. An NLI grounding check compares a claim with a premise, so it needs evidence text. Without retrieval, it has nothing to compare against. In that case, sampling-based detectors such as semantic entropy or SelfCheckGPT are the main option, along with log-probability signals if your API exposes them. A common pattern is to treat a high-uncertainty score as a trigger to run retrieval, then apply the grounding check to the retrieved evidence before answering.
How do you measure hallucination rate in production?
Sample real traces stratified by route, then label them at the claim level against a written rubric. The rubric must state whether correct means faithful to the context or true in the world. Report hallucination rate as unsupported or contradicted claims over total claims, alongside the abstention rate. Evaluate any automated detector against these labels with AUROC, AUPRC and precision at your target recall. Re-measure on every model, prompt or retriever release, because both the error rate and the detector’s calibration shift.
Do guardrails like Bedrock contextual grounding stop all hallucinations?
No. Amazon Bedrock’s contextual grounding check scores whether a response is supported by the grounding source you supply and whether it is relevant to the query. It cannot catch a faithful answer drawn from a wrong or outdated document. It is not designed to judge open-domain facts absent from the source. Its documentation also notes that with streaming, an irrelevant response can reach the user before it is flagged. Treat managed guardrails as one calibrated detector in a layered design, not a guarantee.
Further Reading
- How to add verifiable citations to RAG answers: attribution, span-level grounding and citation evals — the citation side of grounding.
- Fact-checking viral AI hallucination videos and what LLM uncertainty really means — public myths versus mechanisms.
- LLM-as-a-judge evaluation: bias, calibration and pairwise rubrics — when your detector or labeller is itself a model.
- LLM sycophancy: measuring and mitigating agreeable but wrong answers — a related failure that consistency checks also miss.
References
- Farquhar, S., Kossen, J., Kuhn, L., Gal, Y. “Detecting hallucinations in large language models using semantic entropy.” Nature 630, 625–630 (2024). doi:10.1038/s41586-024-07421-0
- Kuhn, L., Gal, Y., Farquhar, S. “Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation.” ICLR 2023. arXiv:2302.09664
- Kossen, J. et al. “Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs.” 2024. arXiv:2406.15927
- Manakul, P., Liusie, A., Gales, M. “SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models.” EMNLP 2023. ACL Anthology
- Wang, X. et al. “Self-Consistency Improves Chain of Thought Reasoning in Language Models.” ICLR 2023. arXiv:2203.11171
- Min, S. et al. “FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation.” EMNLP 2023. arXiv:2305.14251
- Kalai, A. T., Nachum, O., Vempala, S. S., Zhang, E. “Why Language Models Hallucinate.” 2025. arXiv:2509.04664 and OpenAI summary
- Lin, S., Hilton, J., Evans, O. “TruthfulQA: Measuring How Models Mimic Human Falsehoods.” arXiv:2109.07958
- Tang, L., Laban, P., Durrett, G. “MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents.” arXiv:2404.10774
- Niu, C. et al. “RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models.” arXiv:2401.00396
- Ji, Z. et al. “Survey of Hallucination in Natural Language Generation.” ACM Computing Surveys (2022). arXiv:2202.03629
- Kadavath, S. et al. “Language Models (Mostly) Know What They Know.” 2022. arXiv:2207.05221
- Dhuliawala, S. et al. “Chain-of-Verification Reduces Hallucination in Large Language Models.” 2023. arXiv:2309.11495
- Tamber, M. S. et al. “Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards.” 2025. arXiv:2505.04847
- Vectara. HHEM-2.1-Open model card and “Introducing the next generation of Vectara’s hallucination leaderboard” (November 2025).
- Ragas documentation. Faithfulness metric
- Amazon Bedrock User Guide. Use contextual grounding check to filter hallucinations in responses
- NVIDIA NeMo Guardrails. Hallucinations and fact-checking rails
By Riju — about
