How to Add Verifiable Citations to RAG Answers: Attribution, Span-Level Grounding and Citation Evals

A bracketed number after a sentence is not a citation. It is a claim that a citation exists. In most retrieval-augmented generation (RAG) systems nobody checks that claim, and when researchers did check, the results were sobering: on the ELI5 long-form questions in the ALCE benchmark, about half of the answers from the strongest models of the time were not fully supported by the passages they cited. That gap is the whole problem with rag citations as most teams ship them today.

It matters now because citations have moved from a research curiosity to a product requirement. Anthropic, Google and OpenAI all expose citation metadata in their APIs, regulated buyers ask for audit trails, and users have learned to click the little numbers. A citation that points to the wrong passage is worse than no citation, because it borrows credibility it has not earned.

You will leave with a seven-step build recipe, a chunk-ID and output-schema design, a verifier that checks every claim against its cited text, and a runnable evaluation harness that computes citation precision and recall in continuous integration.

What this covers: the AIS definition of attribution, ALCE metrics, cite-as-you-generate versus post-hoc attribution, span-level grounding, vendor citation APIs (dated), UI rendering, an eval harness in Python, and the failure modes that survive all of it.

Context and Background

The research vocabulary for this topic comes from two papers. The first is Rashkin et al., “Measuring Attribution in Natural Language Generation Models” (Google Research, arXiv 2112.12870, later published in Computational Linguistics). It defines Attributable to Identified Sources (AIS): a system output is attributable if a generic reader, shown the source P, would agree that “according to P, s” holds for the statement s. Annotation runs in two stages. First the rater decides whether the output is interpretable on its own; only then do they see the source and judge whether all of the output’s information is supported by it.

The second is Gao, Yen, Yu and Chen, “Enabling Large Language Models to Generate Text with Citations” (EMNLP 2023), which introduced the ALCE benchmark. ALCE turns AIS into automatic metrics. It splits an answer into statements at sentence boundaries, and a natural language inference (NLI) model, TRUE, a T5-11B model fine-tuned on NLI data, checks whether the concatenated cited passages entail each statement. That gives citation recall and citation precision, which we will implement below (ALCE paper on arXiv).

Practitioners have three families of tooling. Retrieval frameworks such as LangChain and LlamaIndex pass source metadata through the pipeline but leave verification to you. Evaluation libraries score faithfulness or groundedness, usually with an LLM judge, at the answer level rather than the claim-to-citation level. And model providers now return structured citation objects: Anthropic’s Citations API (announced January 23, 2025), Gemini’s grounding metadata for Google Search and its File Search tool, and OpenAI’s file_citation annotations in the Responses API.

Our earlier posts cover the retrieval half of the problem. The guide to RAG chunking strategies, hybrid BM25 search and reranking shows how to get the right passages into context, and RAG retrieval evaluation with recall@k, nDCG and golden sets measures whether you did.

This post is about the other half of RAG citations: proving that each sentence the model wrote is supported by the specific passage it points at. My thesis is that a citation is only verifiable if three things are true at once. The pointer is machine-resolvable to an exact span, a deterministic or model-based verifier checks the span against the claim, and the check runs as a regression test, not a one-off audit. Most systems have one of the three. Few have all of them.

How to Add RAG Citations: The Seven-Step Recipe

Direct answer: To add verifiable RAG citations, give every retrieved chunk a stable ID and character offsets, make the model emit claims with chunk IDs and quoted spans in a parseable format, resolve each quote to an exact span in the source, verify each claim against its cited span with an NLI model or judge, then measure citation precision and recall in CI.

Here is the recipe as an ordered list. Each step is expanded below.

Index chunks with stable, resolvable IDs. Store document ID, version hash, chunk ordinal and character offsets for every chunk, so a citation survives re-indexing and points to bytes a human can open.
Pack context with short citation handles. Present retrieved chunks to the model under short labels such as S3, mapped server-side to the full ID, so the model never has to copy a long identifier.
Constrain the output format. Ask for an answer as a list of claims, each with one to three handles and a verbatim supporting quote, using a JSON schema or a tight line-based format.
Resolve quotes to spans. Locate each quote inside its cited chunk with exact then fuzzy matching, and record start and end offsets. Unresolvable quotes are citation hallucinations.
Verify claim-to-span entailment. Run an NLI model, or an LLM judge for hard cases, on each claim against the concatenation of its cited spans, and drop or flag citations that do not entail.
Render citations as inspectable evidence. In the UI, show the quoted span highlighted inside its source, with document version and a deep link, not just a footnote number.
Evaluate continuously. Compute ALCE-style citation recall and precision plus quote-resolution rate on a golden set in CI, and gate releases on them.

Figure 1: Reference pipeline for verifiable RAG citations. Every arrow carries a stable identifier; the verifier sits between generation and rendering, and the eval harness replays the same path offline.

Read the figure top to bottom. The indexer writes chunks with IDs and offsets into the store. At query time, retrieval and reranking select passages, the context packer assigns short handles, and the generator emits claims with handles and quotes. The span resolver maps quotes to offsets, the verifier scores entailment, and only verified citations reach the renderer. The evaluation harness runs the same resolver and verifier over a golden set, which is why the online and offline numbers are comparable.

Step 1 and 2: identifiers are the foundation

Most broken RAG citations are broken at indexing time, long before a model sees anything. If a chunk ID is a random UUID minted on every ingestion run, yesterday’s citation points at nothing after tonight’s re-index. If chunks carry no character offsets, the best you can show the user is “somewhere in this 40-page PDF”, which nobody will check.

A robust ID has four parts: a stable document key (a canonical URL or content-management ID), a content hash of the document version, the chunk ordinal and the half-open character range [start, end) within the normalised document text. A string like kb:pump-manual#v=9f3a1c#c=0042@18230-19410 is ugly, but it is self-describing and survives a database migration.

Do not expose that string to the model. Long opaque IDs waste tokens and models copy them imperfectly; a single transposed hex digit turns a real citation into a dangling one. Instead, number the chunks in the prompt (S1 to S12) and keep the mapping server-side. Short handles are also easier to validate: anything outside the set you issued is, by construction, a hallucinated citation.

Step 3 and 4: claims, quotes and spans

The output format determines what you can verify. Free text with bracketed numbers forces you to guess where each claim begins and ends. A claim list makes the unit of verification explicit. Each claim is one atomic statement, its handles, and a short verbatim quote from each cited chunk that supports it.

Quotes are the cheap trick that makes span-level grounding work. A quote can be checked with a string search, which costs microseconds and has no false positives when it matches exactly. If the model’s quote is not in the cited chunk, you know the citation is wrong before you pay for any NLI call. If it is, you have exact offsets for highlighting.

Quotes have a cost: they add output tokens. Anthropic’s Citations API addresses this directly; its documentation states that the returned cited_text does not count toward output tokens, which makes provider-side quoting cheaper than asking the model to reproduce quotes itself. With other providers, cap quotes at roughly 25 words and accept the token overhead.

Step 5: verification is not optional

Even with exact quotes, the claim can overreach the quote. A chunk that says “the valve is rated to 16 bar” supports “the valve is rated to 16 bar” but not “the valve is safe at 16 bar under all temperatures”. This is the “correct citation, wrong claim” failure, and only a semantic check catches it.

The standard check is NLI: treat the cited span as the premise and the claim as the hypothesis, and accept the citation only if the entailment probability passes a threshold. ALCE used exactly this structure with TRUE. In production, a smaller cross-encoder NLI model handles most claims in milliseconds on a GPU, and an LLM judge handles the residue where the NLI model is uncertain.

Step 6 and 7: rendering and evaluation close the loop

A verified citation shown as a superscript “[3]” still asks the user to do the work. The renderer should open the source at the span, highlight the quoted text, and show which document version it came from. That makes the citation falsifiable by a human in five seconds, which is the practical meaning of “verifiable”.

Evaluation turns all of this into a number that can regress. Without a CI gate, citation quality drifts silently whenever someone changes the chunker, the reranker, the prompt or the model. The harness later in this post computes the metrics on every pull request.

Walk-through: Building Grounded Generation With Span-Level Citations

This section turns the recipe into code. It covers the choice between cite-as-you-generate and post-hoc attribution, the data model, the prompt and parser, the span resolver, the NLI verifier, what the vendor APIs give you for free, and finally the evaluation harness.

Cite-as-you-generate versus post-hoc attribution

There are two ways to attach RAG citations. In cite-as-you-generate, the model sees numbered sources and writes citations inline as it answers. In post-hoc attribution, the model answers first, then a separate stage segments the answer into claims and searches the retrieved passages, or the whole corpus, for support.

Figure 2: Two attribution strategies. Inline citing couples the claim and its evidence in one generation step; post-hoc attribution decouples them and adds a retrieval and NLI pass after generation.

The figure shows the structural difference. Inline citing has one generation call and a light verification pass. Post-hoc attribution adds a segmenter, a per-claim evidence search and an NLI matcher, but it can attach citations to any model’s output, including a model you cannot prompt.

ALCE gives a useful data point here. In its ASQA experiments, a closed-book model with post-hoc citing had citation recall far below the vanilla inline-citing approach; the paper reports it as roughly 47% lower. The intuition is straightforward. A model that wrote from parametric memory produces claims that the retrieved set may simply not contain, so post-hoc search finds nothing to cite. Post-hoc attribution is a repair tool, not a substitute for grounding the generation in the first place.

The practical answer is a hybrid. Generate with inline handles and quotes, then run post-hoc search only for claims whose inline citations fail verification. That second pass rescues claims that were true and supported somewhere in the retrieved set but mis-cited, which is a common failure when two chunks say similar things.

The data model

Start with the chunk record. The fields below are the minimum needed to resolve a citation to an exact, versioned span.

from dataclasses import dataclass
import hashlib

@dataclass(frozen=True)
class Chunk:
    doc_key: str        # canonical source id, e.g. "kb:pump-manual"
    doc_version: str    # sha256 of normalised document text, first 12 hex chars
    ordinal: int        # chunk number within the document
    start: int          # inclusive char offset in normalised doc text
    end: int            # exclusive char offset
    text: str           # the chunk text itself (== doc_text[start:end])
    title: str = ""

    @property
    def chunk_id(self) -> str:
        return f"{self.doc_key}#v={self.doc_version}#c={self.ordinal:04d}@{self.start}-{self.end}"

def doc_version(normalised_text: str) -> str:
    return hashlib.sha256(normalised_text.encode("utf-8")).hexdigest()[:12]

Two details matter. First, offsets refer to a normalised document text, after you strip markup and collapse whitespace, and that same normalised text must be stored or reproducible, otherwise offsets drift. Second, offsets are character offsets in Python string terms, not bytes. Mixing the two silently breaks highlighting for any non-ASCII document, which includes most engineering manuals with degree signs and micro symbols.

Packing context and constraining the output

The prompt maps short handles to chunks and asks for a claim list. JSON with a schema is the most robust format when your provider supports structured outputs. One caveat: Anthropic documents that its Citations feature and structured outputs are incompatible, returning a 400 error if both are enabled. If you use provider-side citations there, use a line-based format instead.

import json

SYSTEM = """You answer strictly from the provided sources.
Return JSON matching this schema:
{"claims": [{"text": str, "cites": [str], "quotes": [str]}],
 "unanswerable": bool}
Rules:
- Each claim is ONE atomic factual statement.
- cites: 1-3 source handles like "S2" that directly support the claim.
- quotes: for each cite, a verbatim excerpt (max 25 words) copied exactly
  from that source, in the same order as cites.
- If the sources do not support an answer, return {"claims": [], "unanswerable": true}.
- Never cite a handle that is not listed."""

def pack_context(chunks: list[Chunk]) -> tuple[str, dict[str, Chunk]]:
    handle_map, blocks = {}, []
    for i, ch in enumerate(chunks, start=1):
        h = f"S{i}"
        handle_map[h] = ch
        blocks.append(f"<source id=\"{h}\" title=\"{ch.title}\">\n{ch.text}\n</source>")
    return "\n\n".join(blocks), handle_map

The “unanswerable” flag is not decoration. Without an explicit abstention path, models fill gaps with plausible text and attach the nearest-looking citation, which is the most damaging citation failure of all. Giving the model a sanctioned way to say “the sources do not cover this” measurably reduces forced citations in my experience, though I have no public benchmark to quote for that effect.

Resolving quotes to spans

The resolver takes each quote and finds it inside the cited chunk. Exact match first, then a whitespace- and case-tolerant match, then a bounded fuzzy match using the standard library’s difflib. A quote that fails all three is a fabricated quote.

import re
from difflib import SequenceMatcher

def _norm(s: str) -> str:
    return re.sub(r"\s+", " ", s).strip().lower()

def resolve_quote(quote: str, chunk: Chunk, min_ratio: float = 0.90):
    """Return (abs_start, abs_end, match_type) in normalised doc offsets, or None."""
    i = chunk.text.find(quote)
    if i >= 0:
        return chunk.start + i, chunk.start + i + len(quote), "exact"
    # tolerant: compare on normalised strings, then map back with a regex
    pattern = r"\s+".join(re.escape(tok) for tok in quote.split())
    m = re.search(pattern, chunk.text, flags=re.IGNORECASE)
    if m:
        return chunk.start + m.start(), chunk.start + m.end(), "normalised"
    # fuzzy: slide a window of the quote's length over the chunk
    q, t = _norm(quote), chunk.text
    best, best_pos = 0.0, None
    w = len(quote)
    step = max(1, w // 4)
    for pos in range(0, max(1, len(t) - w + 1), step):
        r = SequenceMatcher(None, q, _norm(t[pos:pos + w])).ratio()
        if r > best:
            best, best_pos = r, pos
    if best >= min_ratio and best_pos is not None:
        return chunk.start + best_pos, chunk.start + best_pos + w, f"fuzzy:{best:.2f}"
    return None

The fuzzy threshold is a policy decision. At 0.90, the resolver tolerates a dropped article or a changed hyphen, which models produce often, but rejects paraphrase. Paraphrase is not a quote, and accepting it reopens the door to fabricated evidence. Log the match type; a rising share of fuzzy matches after a model upgrade is an early warning.

Verifying claims with NLI

Verification follows the ALCE definitions so that online scores and offline metrics agree. For each claim, concatenate the resolved spans, or the full cited chunks if you prefer recall over strictness, and ask an NLI model whether the premise entails the claim.

from functools import lru_cache
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

NLI_MODEL = "cross-encoder/nli-deberta-v3-base"  # example; swap for your validated model

@lru_cache(maxsize=1)
def _nli():
    tok = AutoTokenizer.from_pretrained(NLI_MODEL)
    mdl = AutoModelForSequenceClassification.from_pretrained(NLI_MODEL).eval()
    labels = {v.lower(): k for k, v in mdl.config.id2label.items()}
    return tok, mdl, labels["entailment"]

def entail_prob(premise: str, hypothesis: str) -> float:
    tok, mdl, ent_idx = _nli()
    enc = tok(premise, hypothesis, truncation=True, max_length=512, return_tensors="pt")
    with torch.no_grad():
        probs = mdl(**enc).logits.softmax(-1)[0]
    return float(probs[ent_idx])

def entails(premise: str, hypothesis: str, tau: float = 0.5) -> bool:
    return entail_prob(premise, hypothesis) >= tau

The model name is an example, not a recommendation backed by a benchmark; validate any NLI checkpoint against a few hundred human-labelled claim and passage pairs from your own corpus before trusting it. Reading the entailment index from id2label rather than hard-coding it avoids a classic bug, because NLI checkpoints disagree on label order.

Note the max_length=512 truncation. If the concatenated premise is longer than the model’s window, the tail is silently dropped and a true citation can fail. Verify against resolved spans plus a small margin of surrounding text, not entire long chunks, which is another reason to insist on quotes.

Figure 3: Per-claim verification sequence. Handle validation and quote resolution are cheap deterministic gates; NLI and the LLM judge run only on claims that pass them.

The ordering in Figure 3 is a cost decision. Handle validation is a dictionary lookup. Quote resolution is a string search. NLI is a forward pass on a small model. The LLM judge is the expensive tail, reserved for claims where NLI probability falls in an uncertainty band, for example between 0.3 and 0.7. On a typical answer of eight claims, most claims should exit at the NLI stage, so the judge runs rarely.

What the vendor APIs give you (as of November 2026)

Provider support for RAG citations removes some of this work, but none of them removes all of it. The table summarises what each returns, verified against provider documentation on the date of writing.

Provider feature	What you send	What comes back	Span granularity	What you still own
Anthropic Citations (announced Jan 23, 2025)	`document` blocks or `search_result` blocks with `citations: {"enabled": true}`	Text blocks carrying `char_location`, `page_location`, `content_block_location` or `search_result_location` citations with `cited_text`	Sentences for plain text and PDF; whole blocks for custom content and search results	Chunk ID mapping, semantic verification, evals
Gemini grounding with Google Search	`google_search` tool	`groundingMetadata` with `groundingChunks` (URI, title) and `groundingSupports` linking a response `segment` (start and end index, text) to chunk indices	Response segment to web source; no quoted source span	Source-side span, verification of web content
Gemini File Search (launched Nov 6, 2025)	Files in a File Search store	Citations in `grounding_metadata`; page-level citations added in a May 2026 update	Chunk and page	Offsets for highlighting, evals
OpenAI Responses `file_search`	Vector store IDs	`file_citation` annotations with `index`, `file_id`, `filename`; raw results only with `include: ["file_search_call.results"]`	A position in the output text and a file, not a source span	Chunk-level mapping, quotes, verification

Three observations follow. Anthropic’s design is the closest to span-level grounding, because cited_text is literally an extract of your document and index conventions are documented: character indices are zero-based with exclusive ends, page numbers one-based with exclusive ends. Its granularity is fixed by its own sentence chunking for text and PDFs, so to cite your RAG chunks precisely, send them as custom content or search_result blocks, each split into small text blocks.

Gemini’s Search grounding annotates the response side very well, with segments of generated text linked to sources, but the source side is a URL, not a span. Google’s documentation does not state whether segment indices are characters or bytes, so test with non-ASCII text before trusting offsets in a multilingual product.

OpenAI’s file_citation tells you which file supported a position in the answer. It does not say which chunk or which characters, so you need the raw search results via the include parameter and your own matching to get to span level. Treat all three feature sets as dated: they changed several times in 2025 and 2026 and will change again.

Citation Precision and Recall: An Eval Harness You Can Run in CI

An eval for RAG citations answers two questions per answer. Is every claim supported by what it cites? Is every citation pulling its weight? ALCE formalised these as citation recall and citation precision, and they remain the clearest pair of metrics for this job.

The metric definitions

Citation recall is computed per statement. A statement scores 1 only if it has at least one citation and the concatenation of all its cited passages entails it. The answer-level score is the mean over statements. A statement with no citation scores 0, which is what you want: uncited factual claims are a recall failure.

Citation precision is computed per citation. ALCE calls a citation irrelevant if it does not entail the statement by itself and the remaining citations still entail the statement without it. A citation scores 1 if its statement has recall 1 and the citation is not irrelevant. The answer-level score is the mean over citations.

The precision definition is subtle and worth internalising. A citation that only partially supports a claim is still counted as precise, as long as removing it would break the entailment. So a claim built from two facts in two passages, each cited, scores full precision. Only padding, a citation that adds nothing, is penalised. That matches how a careful human reviewer thinks about footnotes.

ALCE also caps citations at three per statement, which stops a model from gaming recall by citing everything. Keep that cap; it is cheap insurance.

How well do these automatic metrics track people? The ALCE authors report agreement with human judgments of Cohen’s kappa 0.698 for citation recall and 0.525 for citation precision, with raw agreement of 85.1% and 77.6% respectively. Recall is therefore trustworthy as a regression signal. Precision is noisier, so trend it rather than treating single-run deltas as meaningful.

Two extra metrics production needs

ALCE assumes citations point at passages. Production systems also need to know whether the quotes are real, so add two deterministic metrics.

Handle validity rate: the share of citations whose handle exists in the packed context. Anything below 100% is a bug in the prompt, the parser or the model, and it should block a release outright.
Quote resolution rate: the share of quotes that resolve exactly, normalised or fuzzy, reported per match type. A fall in exact matches is often the first sign that a new model paraphrases inside quote fields.

Add a third, behavioural metric: abstention accuracy on questions the corpus cannot answer. A system with perfect citation precision that answers everything, including questions it has no evidence for, is not safe; it is just good at finding plausible footnotes.

The harness

The harness below consumes a JSONL golden set where each line holds a question, the retrieved chunks at evaluation time and optionally an answerable flag. It runs your pipeline, parses the claim list, resolves quotes, applies the ALCE definitions and emits a JSON report with pass or fail against thresholds. It reuses Chunk, pack_context, resolve_quote and entails from above.

# citation_eval.py  -  ALCE-style citation precision/recall + quote checks
import json, sys, statistics
from dataclasses import dataclass, field
from typing import Callable

MAX_CITES = 3

@dataclass
class ClaimResult:
    text: str
    cites: list[str]
    valid_handles: list[str]
    recall: int = 0
    precise: list[int] = field(default_factory=list)
    quote_matches: list[str] = field(default_factory=list)

def premise_for(handles, handle_map, spans):
    # Prefer resolved spans; fall back to whole chunk text for unresolved quotes.
    parts = []
    for h in handles:
        parts.append(spans.get(h) or handle_map[h].text)
    return " ".join(parts)

def score_claim(claim: dict, handle_map: dict, doc_text_of: Callable[[Chunk], str]) -> ClaimResult:
    cites = claim.get("cites", [])[:MAX_CITES]
    quotes = claim.get("quotes", [])
    valid = [h for h in cites if h in handle_map]
    res = ClaimResult(text=claim["text"], cites=cites, valid_handles=valid)

    spans = {}
    for h, q in zip(cites, quotes):
        if h not in handle_map:
            res.quote_matches.append("bad_handle")
            continue
        hit = resolve_quote(q, handle_map[h])
        if hit is None:
            res.quote_matches.append("unresolved")
            continue
        s, e, kind = hit
        spans[h] = doc_text_of(handle_map[h])[s:e]
        res.quote_matches.append(kind.split(":")[0])

    if not valid:
        res.precise = [0] * len(cites)
        return res

    full = premise_for(valid, handle_map, spans)
    res.recall = int(entails(full, claim["text"]))

    for h in cites:
        if h not in handle_map or not res.recall:
            res.precise.append(0)
            continue
        alone = entails(premise_for([h], handle_map, spans), claim["text"])
        rest = [x for x in valid if x != h]
        rest_ok = bool(rest) and entails(premise_for(rest, handle_map, spans), claim["text"])
        irrelevant = (not alone) and rest_ok
        res.precise.append(0 if irrelevant else 1)
    return res

def evaluate(golden_path: str, run_pipeline: Callable, doc_text_of: Callable,
             thresholds: dict) -> dict:
    claim_recalls, cite_precisions, matches = [], [], []
    abstain_total = abstain_correct = 0
    with open(golden_path) as f:
        for line in f:
            ex = json.loads(line)
            chunks = [Chunk(**c) for c in ex["chunks"]]
            context, handle_map = pack_context(chunks)
            out = run_pipeline(ex["question"], context)   # returns parsed JSON dict
            if ex.get("answerable") is False:
                abstain_total += 1
                abstain_correct += int(bool(out.get("unanswerable")) and not out.get("claims"))
                continue
            for claim in out.get("claims", []):
                r = score_claim(claim, handle_map, doc_text_of)
                claim_recalls.append(r.recall)
                cite_precisions.extend(r.precise)
                matches.extend(r.quote_matches)

    n_q = max(1, len(matches))
    report = {
        "citation_recall": statistics.fmean(claim_recalls) if claim_recalls else 0.0,
        "citation_precision": statistics.fmean(cite_precisions) if cite_precisions else 0.0,
        "handle_validity": 1 - matches.count("bad_handle") / n_q,
        "quote_exact_rate": matches.count("exact") / n_q,
        "quote_resolution_rate": 1 - (matches.count("unresolved") + matches.count("bad_handle")) / n_q,
        "abstention_accuracy": abstain_correct / abstain_total if abstain_total else None,
        "n_claims": len(claim_recalls),
        "n_citations": len(cite_precisions),
    }
    report["pass"] = all(
        report[k] is not None and report[k] >= v for k, v in thresholds.items()
    )
    return report

if __name__ == "__main__":
    from my_rag import run_pipeline, doc_text_of   # your app's entry points
    TH = {"citation_recall": 0.85, "citation_precision": 0.80,
          "handle_validity": 1.0, "quote_resolution_rate": 0.95}
    rep = evaluate(sys.argv[1], run_pipeline, doc_text_of, TH)
    print(json.dumps(rep, indent=2))
    sys.exit(0 if rep["pass"] else 1)

A few design choices deserve explanation. The harness freezes retrieved chunks in the golden set instead of re-running retrieval. That isolates generation and citation quality from retrieval drift; measure retrieval separately with the recall@k harness from our RAG retrieval evaluation guide. Then run a second, end-to-end job with live retrieval so you also see the interaction.

The thresholds in TH are illustrative starting points, not industry standards. Set them from a baseline run: record the current numbers, then fail the build if a change drops recall by more than, say, three points. Handle validity is the exception and should stay at 1.0.

doc_text_of returns the normalised document text for a chunk’s document, so resolved offsets slice the same string the indexer used. If that function and the indexer disagree on normalisation, every exact match turns into a fuzzy match, which the report will make obvious.

A worked example of the arithmetic

Suppose an answer has four claims. Claim A cites S1 and S2, and S1 alone entails it while S2 adds nothing. Claim B cites S3 and S4, each supporting half of the claim, and together they entail it. Claim C cites S5, which does not entail it. Claim D has no citation.

Recall is computed per claim: A scores 1, B scores 1, C scores 0 and D scores 0, so citation recall is 2 of 4, or 0.50. Precision is computed per citation: S1 for A scores 1. S2 for A is irrelevant, since alone it fails and S1 still entails without it, so it scores 0. S3 and S4 both score 1, because neither alone entails B but removing either breaks the entailment. S5 scores 0 because C has recall 0. That gives 3 of 5, or 0.60.

The example shows why both metrics matter. A team optimising only precision could reach 1.0 by deleting every uncertain citation, and recall would collapse. A team optimising only recall could cite three passages per claim, and precision would show the padding.

Cost of running the harness

NLI calls dominate the cost. For a claim with k valid citations, the harness makes one call for the full premise plus, for each citation, up to two more: one alone and one for the rest. That is at most 1 + 2k calls per claim, so 7 calls at the ALCE cap of three citations. A golden set of 300 questions averaging 6 claims and 1.5 citations each needs roughly 300 × 6 × 4 = 7,200 NLI forward passes. A base-size cross-encoder processes that in minutes on one GPU, so the harness fits comfortably in a pull-request job. The figures here are back-of-envelope arithmetic, not measured timings.

If you swap NLI for an LLM judge, the same 7,200 calls become real money and latency. Cache results keyed on a hash of premise and hypothesis; most claims do not change between runs, so the cache hit rate on a stable golden set is high.

To calibrate the judge or the NLI threshold against human labels, the techniques in our post on LLM-as-a-judge evaluation, bias and calibration apply directly: a few hundred labelled pairs, a confusion matrix, and a threshold chosen for the error you care about most.

Rendering RAG Citations Users Will Actually Check

The interface decides whether RAG citations are verifiable in practice. A footnote number that links to a document’s first page is technically a citation, but the reader has to find the supporting sentence alone, and almost nobody does. The renderer’s job is to make checking cheaper than trusting.

Three patterns work. The first is hover or tap previews: the claim’s citation marker opens a card with the highlighted quote, its surrounding sentence, the document title and version. The second is deep links to the span, which open the source viewer scrolled to the offsets with the quote highlighted; for PDFs, map character offsets to a page and a bounding box at indexing time, because computing that in the browser is fragile. The third is visible verification state: claims whose citations failed verification are either removed, rewritten, or shown with a muted “unverified” style rather than a confident footnote.

Grouping matters too. When a single claim cites three sources, show them as one marker with a count, not three adjacent numbers, and order them by entailment score. When several claims cite the same chunk, reuse the same marker number so the reference list stays short.

Finally, store what you rendered. Persist the answer, the claim list, resolved offsets, document versions and verifier scores with the conversation record. When a user disputes an answer weeks later, or an auditor asks why the system said something, you can replay exactly what the model saw and what it pointed at. Our post on LLM observability and LLMOps architecture covers how to attach that payload to traces without blowing up storage.

Trade-offs, Gotchas, and What Goes Wrong

RAG citations fail in recognisable ways. Naming them precisely is the first step to testing for them.

Figure 4: Citation failure modes and where they are caught. Deterministic gates catch fabricated handles and quotes; NLI catches overreach and wrong-passage citations; only evals and humans catch correct-looking citations of wrong or stale sources.

Citation hallucination. The model cites a handle that was never in context, or quotes text that does not exist in the cited chunk. Handle validation and quote resolution catch both deterministically. If either rate drops below 100% in production, treat it as an incident, not noise.

Correct citation, wrong claim. The quote is real and relevant, but the claim says more than the quote: a stronger quantifier, a dropped condition, a merged number. This is the most common failure that survives naive checks, and NLI on resolved spans is the main defence. Watch especially for numbers and units, which small NLI models handle poorly; add a rule that numbers in a claim must appear in its cited spans.

Right claim, wrong passage. The claim is true and supported by another retrieved chunk, but the model cited a neighbour. Users who click through see irrelevant text and lose trust. The hybrid post-hoc pass described earlier repairs many of these by searching the other retrieved chunks for an entailing span.

Citing the stale version. A chunk is real, verified and correct for version 8 of a manual, but the current document is version 9. Version hashes in the chunk ID make this detectable; a nightly job can re-verify stored citations against current versions and flag drift.

Retrieval-bound recall. Citation recall can never exceed what retrieval supplies. ALCE reports that the top five retrieved passages covered only 56.8% of ASQA answers, so even perfect citing would leave large gaps. When citation recall plateaus, look at retrieval before blaming the generator.

Faithful to a wrong source. AIS measures attribution, not truth: the framework asks whether a statement is supported by the identified source, not whether that source is correct. A perfectly cited answer from an outdated forum post is still wrong. Source quality filters belong at indexing time.

Verifier blind spots. NLI models truncate long premises, struggle with tables and multi-hop reasoning, and are miscalibrated on domain jargon. An LLM judge reduces some of these problems but adds cost, variance and its own biases. Neither should be trusted without a labelled calibration set from your own corpus.

Over-abstention. Tighten verification too far and the system refuses or strips most claims. Track answer rate and abstention accuracy beside precision and recall, or you will optimise your way to a system that says nothing.

Multi-hop and aggregated claims

Some answers are not supported by any single span. “Pump P-101 exceeds the plant’s vibration limit” may combine a measurement from a maintenance log with a threshold from a standard. ALCE’s recall definition handles this because it concatenates all cited passages before checking entailment, and its precision definition rewards each necessary citation. Your claim segmentation must keep such statements whole, though. Splitting them into “P-101 vibration is 7.1 mm/s” and “the limit is 4.5 mm/s” loses the inference, and the comparative claim then goes unverified.

Knowledge-graph retrieval makes this harder and more interesting, because the evidence may be a path of triples rather than text. Cite the source passage behind each edge, not the edge itself; our GraphRAG knowledge graph architecture post covers how to keep provenance on extracted triples so that this is possible.

Practical Recommendations

For RAG citations, start with identifiers and evaluation, not with prompts. A beautifully prompted model citing UUIDs that change nightly is unverifiable, and a citation feature without a CI metric will regress the first time someone swaps the reranker. Build the boring plumbing first, then tune generation against numbers.

Prefer provider-native citations when they reach the granularity you need, and wrap them in your own verification layer either way. Anthropic’s search_result blocks map naturally onto RAG chunks; Gemini and OpenAI give you response-side or file-level pointers that need enrichment. Whatever the provider returns, normalise it into one internal citation schema so the renderer and the harness do not care which model produced the answer.

Treat the verifier as a product component with its own tests. Label a few hundred claim and passage pairs from your own documents, measure the NLI model’s precision on “entailed” decisions, and pick a threshold deliberately. Revisit it whenever the domain or the model changes.

Checklist:

[ ] Chunk IDs include document key, version hash, ordinal and character offsets.
[ ] Prompts use short handles; the server maps them back and rejects unknown handles.
[ ] Output is a claim list with handles and verbatim quotes, plus an explicit abstention path.
[ ] Quotes resolve to offsets via exact, normalised and bounded fuzzy matching, with match type logged.
[ ] Every claim passes NLI on its resolved spans; an LLM judge handles the uncertainty band.
[ ] Numbers and units in a claim must appear in its cited spans.
[ ] The UI highlights the quoted span in a versioned source view.
[ ] CI reports citation recall, citation precision, handle validity, quote resolution and abstention accuracy, and fails on regressions.
[ ] Stored citations are re-verified when source documents change version.

Frequently Asked Questions

What is the difference between RAG citations and grounding?

Grounding is the broader property that a model’s answer is based on supplied evidence rather than parametric memory. Citations are the visible, machine-readable pointers that show which evidence supports which part of the answer. A system can be grounded without citations, for example by restricting the model to retrieved context, but users cannot then check it. Citations make grounding auditable. Verifiable citations go one step further: each pointer resolves to an exact span, and a verifier has confirmed that the span actually supports the claim it is attached to.

How do you measure citation precision and recall?

Split the answer into statements, usually sentences or atomic claims. Citation recall is the fraction of statements whose cited passages, concatenated, entail the statement according to an NLI model or a calibrated judge; uncited statements score zero. Citation precision is the fraction of citations that are not irrelevant, where a citation is irrelevant if it does not entail the statement alone and the other citations still entail it without it. These are the definitions from the ALCE benchmark (Gao et al., EMNLP 2023), and they reward necessary citations while penalising padding.

What does Attributable to Identified Sources mean?

Attributable to Identified Sources, or AIS, is the evaluation framework from Rashkin et al. at Google Research. A system statement is attributable if it is interpretable on its own and a generic reader, shown the identified source, would agree that “according to the source, the statement holds”. Human annotation runs in two stages: first interpretability without the source, then attribution with it. AIS separates attribution from truth. It asks whether the source supports the statement, not whether the source itself is correct, so source quality still needs separate control.

Should I use inline citations or post-hoc attribution?

Use inline citations as the default, because the model writes claims while looking at numbered evidence, which keeps generation and attribution coupled. ALCE found post-hoc citing of closed-book answers produced much lower citation recall on ASQA than inline citing. Post-hoc attribution is still valuable as a repair pass: run it on claims whose inline citations fail verification, searching the other retrieved chunks for an entailing span. It is also the only option when you must attach citations to output from a model you cannot prompt.

Do provider citation APIs make verification unnecessary?

No. Provider features reduce plumbing, and Anthropic’s Citations API in particular returns real extracted text with documented offsets. But a returned citation shows where the model says its evidence is, not that the evidence entails the sentence. Gemini’s Search grounding links response segments to web sources without source spans, and OpenAI’s file search annotations point to files rather than passages. You still need chunk mapping, semantic verification for overreach, and an evaluation harness. Treat provider citations as a high-quality input to your verifier.

Which NLI model should verify LLM citation attribution?

ALCE used TRUE, a T5-11B model fine-tuned on NLI data, which is accurate but heavy for online use. Many teams run a smaller cross-encoder NLI model online and escalate uncertain cases to an LLM judge. No public benchmark settles which checkpoint is best for your documents, so label a few hundred claim and passage pairs from your corpus and measure precision on entailment decisions. Watch for premise truncation, tables, numbers with units and domain jargon, which are where smaller NLI models fail most often.

How to Add Verifiable Citations to RAG Answers: Attribution, Span-Level Grounding and Citation Evals (2026)