GraphRAG: Knowledge-Graph Retrieval Architecture for 2026

Ask a naive vector-RAG system “what are the recurring themes across these 4,000 incident reports?” and it will retrieve the twelve chunks whose embeddings sit nearest your query — then confidently summarize those twelve as if they were the whole corpus. The answer is fluent, plausible, and blind to everything it did not retrieve. This is the structural failure that GraphRAG was built to fix. Instead of treating a corpus as a bag of independent text chunks, GraphRAG extracts the entities and relationships buried in that text, assembles them into a knowledge graph, clusters the graph into communities, and pre-summarizes each community so the system can reason about the whole rather than a lucky sample of the parts.

That structure is not free. Graph indexing can cost an order of magnitude more than embedding the same corpus, and it introduces failure modes — extraction errors, graph drift, cost blowups — that flat vector RAG never had. This post is a practitioner’s map of when the trade is worth it.

What this covers: the naive-vector-RAG failure that motivates graphs, the full GraphRAG indexing pipeline, query-time global vs local search, hybrid vector-plus-graph retrieval, a worked cost and latency analysis, the real failure modes, a decision matrix, and a clear rule for when not to reach for a graph.

Context and Background

Retrieval-augmented generation became the default enterprise pattern for grounding large language models around 2023: chunk your documents, embed each chunk, store the vectors, and at query time pull the top-k nearest neighbours into the prompt. It works remarkably well for one class of question — “find me the passage that answers this” — where the answer lives inside a single, semantically-similar span of text. For a deeper treatment of when to reach for retrieval versus other grounding strategies, see our comparison of fine-tuning vs RAG vs long context.

The trouble starts when the answer is not in any single chunk but spread across many, or when it depends on relationships between facts rather than the facts themselves. “Which suppliers are two hops away from a sanctioned entity?” is not answerable by nearest-neighbour lookup — the relevant chunks may share no vocabulary with the query at all. “Summarize the whole dataset’s position on X” is a global question, and vector search is fundamentally a local tool.

Microsoft Research crystallized the alternative in its April 2024 paper From Local to Global: A Graph RAG Approach to Query-Focused Summarization, and shipped an open-source implementation shortly after. The core claim, backed by their evaluation, is that a graph index plus community summaries delivers materially better comprehensiveness and diversity on global sensemaking questions than a vector baseline. Parallel ecosystems grew around Neo4j, LlamaIndex, and LangChain, each offering knowledge-graph construction and traversal as first-class RAG primitives. By 2026 the practitioner consensus has hardened into a pattern rather than a product: use vectors for semantic entry points, use graphs for relational depth.

GraphRAG Architecture: The Indexing Pipeline

GraphRAG splits into two halves with very different cost profiles: an expensive, LLM-heavy indexing phase run once (and refreshed periodically), and a cheaper query phase run per request. Understanding the indexing pipeline is the key to understanding both the capability and the cost.

In short: GraphRAG indexing chunks your documents, uses an LLM to extract entities and relationships from each chunk, merges duplicate entities into a single knowledge graph, runs Leiden community detection to cluster related entities hierarchically, and then generates an LLM summary for every community — producing a graph plus a set of pre-computed summaries that query time can exploit.

Figure 1: The GraphRAG indexing pipeline — text units flow through LLM extraction into a resolved knowledge graph, which is clustered by Leiden into hierarchical communities and summarized.

Figure 1 traces the flow left to right. Source documents are split into text units, an LLM extracts entities and relations from each unit, duplicate entities are resolved into single nodes, the resulting graph is clustered into communities, and each community is summarized. Node and edge embeddings are computed in parallel so that query-time retrieval can mix semantic and structural signals. The output is a graph index store holding four artifacts: the graph itself, the community hierarchy, the per-community summaries, and the embeddings.

Chunk, then extract entities and relations

The pipeline begins conventionally: documents are split into text units, typically 300 to 1,200 tokens each. Chunk size matters more here than in vector RAG because it is the unit of extraction, not just retrieval. Larger chunks give the extraction LLM more context to resolve pronouns and implicit relationships, but risk diluting attention across too many entities; smaller chunks extract cleaner triples but lose cross-sentence relationships.

The entity extraction step is where GraphRAG earns its cost. For each text unit, an LLM is prompted to emit a structured list of entities (with type and description) and relationships between them — effectively subject-predicate-object triples such as Acme Corp — supplies — Battery Module 7. This is a full LLM call per chunk, and it is the single largest line item in the indexing budget. Some implementations add a second “gleaning” pass that re-prompts the model to catch entities it missed, doubling extraction cost in exchange for recall.

Resolve entities and build the graph

Extraction produces the same real-world entity under many surface forms: “Acme”, “Acme Corp”, “Acme Corporation”. Entity resolution merges these into a single canonical node, unioning their descriptions and re-pointing their edges. Do this badly and the graph fractures — one company becomes five disconnected nodes, and multi-hop traversal silently fails to connect facts that belong together. Resolution strategies range from exact-match and embedding-similarity clustering to a further LLM adjudication pass on ambiguous pairs; the choice trades indexing cost against graph quality.

The resolved entities become nodes and the relationships become edges. Each node carries a description synthesized from every mention, source-chunk references so answers can be cited, and an embedding vector so the node is reachable by semantic search. The edges carry their own descriptions and a weight reflecting how often the relationship was observed.

Detect communities and summarize them

A raw entity graph is still too granular to answer global questions — you cannot stuff ten thousand nodes into a prompt. GraphRAG runs the Leiden algorithm, a modularity-optimizing community-detection method, to partition the graph into clusters of densely-connected entities. Crucially it does this hierarchically: top-level communities are recursively subdivided into sub-communities until leaf communities are small enough to summarize directly. A community might correspond to “the North American logistics network” at one level and “cold-chain carriers in Texas” at a deeper level.

Then comes the second big LLM expense: community summarization. For every community at every level of the hierarchy, an LLM writes a report-style summary of the entities and relationships it contains. These summaries are the secret weapon of global search — they let the system reason about broad themes without re-reading the raw corpus. They are also why full GraphRAG indexing is expensive: a corpus can yield thousands of communities, each demanding its own LLM call.

Query Time: Global vs Local Search, and Hybrid Retrieval

Indexing built the structure; query time exploits it. GraphRAG offers two distinct retrieval modes, and choosing between them — or routing automatically — is the central query-time decision.

Figure 2: Query routing in GraphRAG — broad thematic questions take the global map-reduce path over community summaries, while specific-entity questions take the local traversal path.

Figure 2 shows a router classifying the incoming query. Broad, thematic questions are sent to global search, which fans the query out across community summaries in a map step, generates partial answers, then reduces them into a single response. Specific-entity questions go to local search, which finds the entry entities, walks their neighbourhood, gathers the associated text units, and assembles a focused local context. Both paths converge on a final answer.

Global search — map-reduce over community summaries

Global search answers “what does the whole corpus say about X?” It does not touch the raw text. Instead it selects community summaries at a chosen hierarchy level, and in a map phase asks the LLM to answer the query against each summary while scoring how relevant that summary’s contribution is. A reduce phase then combines the highest-scoring partial answers into a final response. Because the map phase can run over hundreds of community summaries, global search is token-hungry and higher-latency — but it is the only mode that genuinely reasons about the entire dataset rather than a retrieved sample. This is the mechanism behind GraphRAG’s comprehensiveness advantage on sensemaking questions.

Local search — entity-anchored traversal

Local search answers “tell me about this specific thing and what connects to it.” It embeds the query, finds the most relevant entity nodes as entry points, then traverses outward to neighbouring entities, their relationships, the community summaries they belong to, and the original text units that mentioned them. All of this is ranked and packed into the context window up to a token budget. Local search is cheaper and faster than global search and directly supports multi-hop questions — the kind where the answer requires chaining “A relates to B relates to C” and where flat vector RAG, retrieving A, B, and C’s chunks independently, often fails to connect them.

Hybrid vector-plus-graph retrieval

In production, the sharpest systems in 2026 do not choose graph or vector — they fuse them. Hybrid retrieval uses vector search for what it is best at (semantically locating relevant entry points, even when phrasing differs) and graph traversal for what it is best at (following explicit relationships several hops deep).

Figure 3: Hybrid retrieval — vector search supplies semantic entry points while entity linking seeds graph traversal; the two result sets are merged, reranked, and pruned to a token budget before generation.

Figure 3 shows the fusion. The query runs two retrievals in parallel: a vector search for the top-k semantically similar chunks, and entity linking that seeds a multi-hop graph traversal. The semantic entry points and the traversal results are merged, reranked (often with a cross-encoder or reciprocal-rank fusion), and pruned to fit the token budget before the context pack reaches the LLM. The vector side guards against extraction gaps — if an entity was never extracted, semantic search can still surface its chunk — while the graph side supplies the relational depth vectors cannot. This complementarity is why hybrid has become the default enterprise posture rather than pure GraphRAG.

Deeper Analysis: Cost, Latency, and a Worked Trade-off

The single most important thing to internalize about GraphRAG is that its costs and benefits land at different times. The benefit shows up at query time as better answers on relational and global questions. The cost shows up at indexing time, and it is large.

Published figures put full GraphRAG indexing at roughly 10 to 40 times the cost of embedding the same corpus for vector RAG — one widely-cited breakdown gives a range of $20 to $500 for a typical corpus versus $2 to $5 for the vector equivalent. The gap is entirely explained by LLM calls: entity extraction is one call per chunk (two with gleaning), and community summarization is one call per community across every level of the hierarchy. Vector RAG makes zero generative LLM calls at index time — it only runs the far cheaper embedding model.

A worked cost-and-latency reasoning passage

Consider a 50,000-chunk corpus and treat the numbers as illustrative order-of-magnitude reasoning rather than a quote for any specific model or provider.

For vector RAG, indexing is one embedding call per chunk. Embeddings are cheap — often a small fraction of a cent per thousand tokens — so 50,000 chunks might cost a few dollars and finish in minutes. Query-time retrieval is a single approximate-nearest-neighbour lookup, typically single-digit to low-tens of milliseconds, plus one generation call.

For full GraphRAG, indexing runs an extraction LLM call over all 50,000 chunks. If extraction averages, say, 1,500 input plus output tokens per chunk against a mid-tier model, you are paying for 75 million tokens on extraction alone — and that is before entity resolution, before embeddings, and before community summarization adds thousands more generation calls. This is precisely how a corpus that costs a few dollars to embed can cost hundreds of dollars to graph-index, and why indexing is measured in hours, not minutes.

At query time the ordering can invert. Local search is comparable to vector RAG — one traversal plus one generation. But global search runs a map step across many community summaries; if it maps over 200 summaries, that is 200 LLM calls fanned out, then a reduce call. Global queries are therefore the expensive, higher-latency mode, and you should route to them deliberately, not by default.

Two numbers reshaped this calculus in 2025. First, reported evaluations credit GraphRAG with meaningful quality gains on multi-hop and global questions — on the order of a double-digit-percentage lift in comprehensiveness and recall over vector baselines, though exact figures depend heavily on dataset and judge. Treat any single headline percentage as directional. Second, Microsoft Research’s LazyGraphRAG (June 2025) attacks the cost head-on by deferring all LLM summarization to query time and doing only lightweight graph construction during indexing — reported to bring indexing cost down toward vector-RAG levels while preserving much of the global-query quality. If cost is your blocker, the lazy variant changes the decision.

A decision matrix

The honest answer to “GraphRAG or vector RAG?” is “it depends on your question distribution and your budget.” The matrix below is the compressed version.

Dimension	Vector RAG	GraphRAG (full)	Hybrid vector + graph
Best question type	Single-passage factoid lookup	Global sensemaking, multi-hop relational	Mixed workloads, most enterprise apps
Indexing cost	Lowest (embeddings only)	Highest (10-40x; LLM extraction + summarization)	High (graph build + embeddings)
Indexing time	Minutes	Hours	Hours
Query latency	Lowest	Local low, global high	Moderate; depends on routing
Multi-hop reasoning	Weak	Strong	Strong
Whole-corpus questions	Cannot answer well	Strong (global search)	Strong
Citations / provenance	Chunk-level	Entity + chunk-level	Entity + chunk-level
Operational complexity	Low	High (graph store, resolution, refresh)	Highest
Failure mode	Missed context	Extraction errors, graph drift	Both, plus fusion tuning

Read the matrix as a routing guide, not a verdict. If nine in ten of your queries are single-passage lookups, the graph rarely pays for itself. If your users ask relational and thematic questions, or you are building agent memory that must chain facts, the graph earns its complexity. For agentic systems specifically, where the retriever is a tool the agent calls repeatedly, see our breakdown of agentic RAG architecture patterns.

The end-to-end reference architecture

Assembling the pieces gives a system with two clearly separated planes: an offline indexing plane and an online serving plane. Figure 4 shows how they connect in a production deployment.

Figure 4: A production GraphRAG reference architecture — the indexing pipeline populates a knowledge graph, vector store, and community summaries, while the serving plane routes queries through a hybrid retriever to the generator, with an evaluation and observability loop feeding back to the query API.

The indexing pipeline writes three stores — the knowledge graph, the vector store, and the community summaries — each optimized for a different retrieval mode. On the serving side, the query API hands each request to the router, which drives the hybrid retriever across all three stores before the generator produces an answer. The often-omitted component is the evaluation-and-observability loop: without it you cannot see extraction quality regressions, graph drift, or a global-search cost spike until users complain. Treat observability as a first-class store, not an afterthought bolted on later. Log which retrieval mode served each query, how many community summaries a global search touched, and the token cost per answer — those three signals catch most of the failure modes below before they compound.

Trade-offs, Gotchas, and What Goes Wrong

GraphRAG’s structure introduces failure modes that flat vector RAG simply does not have. Knowing them is the difference between a graph that compounds value and one that quietly corrupts every answer.

Extraction errors propagate. The graph is only as good as the LLM extraction that built it. A missed relationship means a hop that can never be traversed; a hallucinated one means a false edge that will confidently mislead. Because extraction runs chunk-by-chunk, errors are silent and distributed — you will not see them until an answer traverses a bad edge. Sampling extracted triples for manual review, and cross-checking with the vector side, is the practical mitigation.

Entity resolution is a two-sided trap. Under-merging fractures one entity into several disconnected nodes, breaking multi-hop paths. Over-merging fuses distinct entities — two different people named “J. Smith” — and creates false connections that leak information across unrelated subgraphs. Both are subtle and both degrade traversal quietly.

Graph drift is the maintenance tax. Your corpus is not static. New documents introduce new entities and relationships, and worse, may contradict old ones. A naive re-index is expensive; an incremental update risks inconsistency between old and new subgraphs. Without a refresh strategy, the graph slowly diverges from reality — the retrieval equivalent of stale cache. Budget for re-indexing cadence up front.

Cost blowups come from global search and gleaning. The two easiest ways to burn budget are routing everything to global search (fanning LLM calls across hundreds of summaries per query) and enabling aggressive multi-pass gleaning at index time. Both are defensible in isolation and ruinous by default. Meter them.

The plateau is real. Reported evaluations suggest graph-traversal advantages plateau as corpora grow very large — past roughly the low tens of millions of tokens, traversal becomes less discriminative because too many entities are reachable within a few hops. Bigger is not automatically better; a sprawling graph can dilute the very signal it was built to concentrate.

Practical Recommendations

Start by profiling your query distribution before writing a line of graph code. Sample real user questions and label each as single-passage factoid, multi-hop relational, or global sensemaking. If the first bucket dominates, ship vector RAG and stop — the graph will not pay for itself. If the second and third buckets are material, GraphRAG or hybrid is justified.

When you do build a graph, default to hybrid retrieval with a router, not pure GraphRAG. Let vector search cover semantic entry points and extraction gaps; let the graph supply relational depth; classify each query and send global-sensemaking questions to global search and everything else to local or vector. This is the pattern the field converged on for good reason. GraphRAG shares its routing DNA with self-correcting retrieval loops — see our guide to corrective RAG and self-RAG patterns for the evaluator side of that design.

A pre-flight checklist before you commit to a graph index:

[ ] Profiled the query mix; relational or global questions are a real share.
[ ] Estimated indexing cost from chunk count × extraction cost, plus community summarization; confirmed the budget.
[ ] Chosen an entity-resolution strategy and a plan to audit sampled triples.
[ ] Decided a re-index / incremental-update cadence to control graph drift.
[ ] Metered global search and gleaning so they cannot silently blow the budget.
[ ] Evaluated LazyGraphRAG or a hybrid variant if full-index cost is the blocker.
[ ] Kept a vector-RAG baseline running to A/B answer quality honestly.

Frequently Asked Questions

What is GraphRAG and how is it different from normal RAG?

GraphRAG is a retrieval-augmented generation pattern that builds a knowledge graph from your documents instead of treating them as independent text chunks. It uses an LLM to extract entities and relationships, clusters them into communities, and pre-summarizes those communities. Normal (vector) RAG retrieves the top-k chunks nearest your query embedding. The key difference: GraphRAG can answer relational, multi-hop, and whole-corpus questions that vector RAG structurally cannot, at the cost of a much more expensive indexing phase.

When should I NOT use GraphRAG?

Skip GraphRAG when your queries are overwhelmingly single-passage factoid lookups where the answer lives in one chunk — vector RAG is cheaper, faster, and simpler for that. Also avoid it when your indexing budget cannot absorb 10-40x the vector cost, when your corpus changes so fast that graph drift outpaces re-indexing, or when you lack the operational maturity to run a graph store, entity resolution, and refresh pipelines. A weak graph gives worse answers than a good vector index.

How much does GraphRAG indexing actually cost?

Published breakdowns put full GraphRAG indexing at roughly 10 to 40 times the cost of embedding the same corpus — on the order of tens to hundreds of dollars for a typical corpus versus a few dollars for vector RAG. The cost is dominated by LLM calls: one entity-extraction call per chunk (two with gleaning) plus one summarization call per community across the hierarchy. LazyGraphRAG (June 2025) defers summarization to query time and reportedly brings indexing cost close to vector-RAG levels.

What is the difference between global and local search in GraphRAG?

Global search answers broad, thematic questions about the entire corpus by running a map-reduce over community summaries — it reasons about the whole dataset but is token-hungry and higher-latency. Local search answers specific-entity questions by finding entry entities and traversing their neighbourhood to gather related facts and source text. Local search is cheaper and directly supports multi-hop reasoning. A router should classify each query and pick the mode; global should be used deliberately, not by default.

What is hybrid vector-plus-graph retrieval?

Hybrid retrieval runs vector search and graph traversal together rather than choosing one. Vector search finds semantically relevant entry points even when wording differs, and it covers gaps where an entity was never extracted. Graph traversal follows explicit relationships several hops deep for relational depth. The two result sets are merged, reranked, and pruned to a token budget before generation. By 2026 this is the default enterprise posture because it combines the strengths of both while hedging their respective failure modes.

What are the main failure modes of GraphRAG?

Four dominate. Extraction errors create missing or false edges that silently mislead traversal. Entity-resolution mistakes either fracture one entity into several nodes (breaking multi-hop paths) or fuse distinct entities (creating false connections). Graph drift accumulates as the corpus changes and the index goes stale. Cost blowups come from over-routing to global search and from aggressive gleaning. All four are quiet failures — they degrade answers without throwing errors — so sampling, auditing, and A/B against a vector baseline are essential.

GraphRAG: Knowledge-Graph Retrieval Architecture (2026)