Document AI OCR in 2026: Pipelines vs Vision-Language Models

Most retrieval-augmented generation (RAG) systems fail long before the language model gets involved. They fail at the PDF. A two-column paper gets read straight across the gutter, a financial table collapses into a stream of numbers with no headers, and a scanned contract produces plausible-looking text with a few silently wrong digits. Garbage in the index means confident garbage in the answer.

In 2026 the choice for document AI OCR is no longer “Tesseract or a cloud API”. You can run a modular pipeline of small specialised models (Docling, MinerU, Marker), an end-to-end vision-language model (VLM) such as olmOCR or DeepSeek-OCR, or a hosted service such as Mistral OCR. Each makes a different bet about where errors should be allowed to occur.

You will leave with a mental model of how each family works, verified benchmark numbers with their caveats, a cost model you can adapt, a routing design, and runnable Docling code for PDF parsing for RAG.

What this covers: the pipeline versus end-to-end split, what the published benchmarks do and do not tell you, hallucination as a new failure class, chunking, evaluation, and a decision matrix.

Context and Background

Optical character recognition (OCR) used to mean one thing: turn pixels into characters. Engines such as Tesseract (originally from HP, later maintained by Google, now a community project) segment a page into lines, then recognise characters or sequences with a neural network. They return text plus bounding boxes and confidences. They do not know that a block is a table, that a footnote belongs at the bottom, or that two columns should be read one after the other.

“Document AI” is what you get when you add that knowledge. The field has split into three families.

Classical and modular pipelines. A layout model first finds regions (text, title, table, figure, formula), then specialised models handle each region: an OCR engine for text lines, a table-structure model for grids, a formula model for LaTeX. Docling from IBM Research is the best-known open example. Its technical report describes layout analysis built on a model trained on DocLayNet and table structure recognition with TableFormer, packaged as an MIT-licensed library that runs on commodity hardware. Marker, MinerU and Unstructured follow broadly similar designs with different model choices.

End-to-end VLM OCR. A single vision-language model looks at a page image and emits Markdown, HTML tables and LaTeX directly. olmOCR from the Allen Institute for AI (Ai2) fine-tunes a 7B-parameter VLM for this. Its paper reports converting a million PDF pages for about 176 USD, against over 6,240 USD for GPT-4o on the same workload. DeepSeek-OCR, Nanonets-OCR2, PaddleOCR-VL and Chandra belong here too.

Hosted document APIs. Mistral OCR 3 (API model mistral-ocr-2512) returns Markdown with HTML table tags that retain colspan and rowspan. Cloud vendors offer comparable services, and general multimodal models from OpenAI, Anthropic and Google can read page images directly.

The practical question is not which family wins. It is which family fails in the way your downstream system can tolerate. We covered how page images can bypass parsing entirely in ColPali and visual document retrieval; this post is about the case where you still need clean text, tables and structure, because you must filter, cite, audit or extract fields.

For standards-minded readers, note what is not standardised here. There is no equivalent of an RFC for “correct Markdown from a PDF”. The closest we have are public benchmarks, which we examine below, and every one of them embeds editorial choices about what counts as correct.

Two Architectures, Two Error Models

Direct answer: a pipeline decomposes a page into regions and routes each to a specialist model, so errors are local and inspectable. An end-to-end VLM generates the whole page as one token sequence, so it handles messy layouts gracefully but can hallucinate or loop. Pick by which error type you can detect and recover from.

The modular pipeline

Figure 1: A modular document AI OCR pipeline. Layout detection fans out to specialist models, and reading-order assembly joins the results with provenance.

Figure 1 shows the shape. The page is rendered to an image (or, for born-digital PDFs, text and coordinates are read straight from the file). A layout detector proposes labelled boxes. Text boxes go to an OCR engine, table boxes to a structure model, formula boxes to a math recogniser. A final stage orders everything and serialises it.

The most important property is provenance. Every output element carries a page number and bounding box. Docling’s unified DoclingDocument representation keeps that structure and exports to Markdown, HTML, DocTags and lossless JSON. When a user disputes an answer, you can highlight the exact region on the source page. That is hard to retrofit onto a model that only emits text.

The second property is that you can swap parts. If your corpus is Arabic invoices, replace the OCR engine without touching layout. If the table model is the weak link, upgrade only that. The cost is engineering: several models, several failure surfaces, and error compounding. A layout box that clips a line by a few pixels produces a truncated string, and no later stage will notice.

Classical engines also have a trait that is easy to undervalue: they cannot invent text. Tesseract may read a “1” as an “l”, but it will not produce a sentence that is not on the page. Its errors are character-level and usually visible in a confidence score.

The end-to-end VLM

Figure 2: End-to-end vision language model OCR. A page image goes in, structured text comes out, and a validation gate decides whether to retry or escalate.

An end-to-end model removes the stage boundaries. A vision encoder turns the page into visual tokens, a language model decodes Markdown. Layout, reading order and table structure are all learned implicitly. This is why VLMs do well on multi-column pages, academic papers with equations, and degraded scans where a rigid layout detector gives up.

olmOCR’s repository describes the current release line (v0.4.0, October 2025) as a model “based on a 7B parameter VLM”, trained further with reinforcement learning, and servable through vLLM or any OpenAI-compatible endpoint. The model card for IBM’s Granite-Docling-258M shows the small end of the range: built on Idefics3 with a SigLIP2 vision encoder and a Granite 165M language model, released September 2025 under Apache 2.0. It emits a markup called DocTags, which is then converted to a DoclingDocument.

The error model differs completely. A VLM is a generator. When the page is blurry it does not return low confidence; it returns fluent text. It can skip a paragraph, duplicate a row, or fall into a repetition loop. The Granite-Docling card itself lists “fewer infinite loops” as an improvement over its predecessor and warns that on GPUs without bfloat16 support the output can degrade into repeated characters. That is a failure class pipelines largely do not have.

Visual-token compression changes the economics

DeepSeek-OCR (released October 2025, MIT licence) reframed the problem as “contexts optical compression”. Its paper reports roughly 97% OCR precision when the number of text tokens is within 10 times the number of vision tokens, falling to about 60% at roughly 20 times compression. It also claims to beat GOT-OCR2.0 on OmniDocBench using 100 vision tokens against 256, and MinerU2.0 using fewer than 800 against more than 6,000 on average. The README lists resolution modes from Tiny (512 by 512, 64 vision tokens) to Large (1280 by 1280, 400 tokens), plus a dynamic “Gundam” mode. These are the authors’ own figures from a preprint, so treat them as vendor claims until reproduced; the independent olmOCR-Bench run below is a useful cross-check.

The engineering lesson is that VLM OCR cost is dominated by the visual token budget per page and the decoded output length. Both are knobs. Halving the resolution roughly halves the encoder cost but eats directly into the small-text accuracy.

Why “hybrid” is the practical answer

There is a third design that is quietly winning in production: use a pipeline to structure the document and a small VLM only on regions that need it. Docling supports this directly. Its README shows a VLM pipeline option (--pipeline vlm --vlm-model granite_docling) alongside the default pipeline, both producing the same DoclingDocument. Because the output type is shared, you can route per document without changing the downstream chunker or index.

What the Benchmarks Actually Say

Three public benchmarks matter, and each measures something different. Reading them side by side avoids the most common mistake, which is to quote a leaderboard rank as if it were a property of your data.

olmOCR-Bench

olmOCR-Bench, from Ai2, scores models on a set of pass/fail unit tests over pages grouped into categories such as ArXiv, old scans, old scans with math, tables, headers and footers, multi-column and long tiny text. A test might assert that a particular sentence is present, that a header is absent from the body, or that two cells appear in the right relationship. Here are numbers copied from the olmOCR README at the time of writing (overall score with reported confidence interval):

System	Tables	Old scans	Multi column	Overall
Mistral OCR API	60.6	29.3	71.3	72.0 ± 1.1
Marker 1.10.1	72.9	33.5	80.0	76.1 ± 1.1
MinerU 2.5.4	84.9	33.7	78.2	75.2 ± 1.1
DeepSeek-OCR	80.2	33.3	66.4	75.7 ± 1.0
Nanonets-OCR2-3B	86.8	40.9	81.9	69.5 ± 1.1
PaddleOCR-VL	84.1	37.8	79.9	80.0 ± 1.0
Chandra OCR 0.1.0	88.0	50.4	81.2	83.1 ± 0.9
olmOCR v0.4.0	84.9	47.7	83.7	82.4 ± 1.1

Several things stand out. The top three overall scores (Chandra, olmOCR, an Infinity-Parser entry in the full table) sit within about one point, which is inside the confidence interval, so their ordering is not meaningful. Old scans are hard for everyone: the best score in that column is about 50, and the mistral entry is under 30. And the benchmark is produced by one of the contenders, so the test design may favour its strengths. The README also marks some rows as results taken from other sources rather than rerun, which weakens direct comparison.

Note also that the Mistral figure here is for the “Mistral OCR API” as tested by Ai2, not necessarily Mistral OCR 3. Mistral’s own announcement for OCR 3 reports a “74% overall win rate over Mistral OCR 2” on forms, scans, complex tables and handwriting using internal, customer-based test sets and a fuzzy-match metric. That is a relative claim on a private set; it cannot be compared to the table above.

OmniDocBench

OmniDocBench (CVPR 2025, Apache 2.0, from OpenDataLab) is larger and more structured: 1,651 PDF pages across 10 document types, 5 layout types and 5 language types, annotated with 28 block-level and 4 span-level element types, including LaTeX for formulas, LaTeX and HTML for tables, and reading order. Its end-to-end Overall score is ((1 minus text edit distance) times 100, plus table TEDS, plus formula CDM) divided by 3. TEDS is Tree Edit Distance Similarity for tables; CDM is Character Detection Matching for formulas.

The leaderboard (the README labels it v1.6 full) is led by the PaddleOCR-VL family and MinerU2.5-Pro:

Method	Overall	Text edit (lower is better)	Formula CDM	Table TEDS
PaddleOCR-VL-1.6	96.34	0.0326	97.53	94.76
MinerU2.5-Pro	95.75	0.036	97.45	93.42
GLM-OCR	95.22	0.044	97.18	92.83
PaddleOCR-VL-1.5	94.93	0.038	96.89	91.67
PaddleOCR-VL	94.18	0.040	95.91	90.65

Two readings are worth making. First, the leading systems on both benchmarks are small, specialised VLMs, and some are hybrids (MinerU is a pipeline with VLM components). The old framing of “pipeline versus VLM” is blurring. Second, the OmniDocBench overall mixes three different metrics into one number, so a system strong at text and weak at formulas can tie with one that is the reverse. If your corpus has no equations, the formula term is irrelevant to you and the overall score misleads. Download the per-attribute breakdown instead.

What no benchmark covers

Neither benchmark measures your scanner, your fonts, your languages, or your tolerance for a wrong digit. Neither measures speed or cost. Neither measures the failure we care about most in regulated settings: a plausible wrong value. A text edit distance of 0.03 sounds excellent, yet on a 3,000-character page it still allows around a hundred character edits, and if those land in an invoice total they matter more than the other 2,900 characters combined. We return to this in the evaluation section.

Walk-through: PDF Parsing for RAG with Docling

Because the output of any parser is only useful once it is chunked, embedded and retrievable, here is an end-to-end path with Docling. The code reflects the Docling API as of the documentation I checked; option class names and chunker constructor arguments have changed across releases, so pin your version and consult the current docs and example notebooks before relying on it.

# pip install docling
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions

opts = PdfPipelineOptions()
opts.do_ocr = True                 # OCR only where there is no text layer
opts.do_table_structure = True     # run the table structure model

converter = DocumentConverter(
    format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=opts)}
)

result = converter.convert("https://arxiv.org/pdf/2408.09869")
doc = result.document                     # a DoclingDocument
print(doc.export_to_markdown()[:800])     # human-readable view
doc_json = doc.export_to_dict()           # lossless view with provenance

The minimal form, taken straight from the project README, is just DocumentConverter().convert(source) followed by export_to_markdown(). The options above add control. Two points matter in production. Run do_ocr only on pages that need it, because full-page OCR on a born-digital PDF throws away a perfect text layer and adds errors. And keep the lossless JSON, not only Markdown, because Markdown cannot hold bounding boxes.

Switching to the VLM path uses the same converter surface. From the CLI the README documents:

docling --pipeline vlm --vlm-model granite_docling ./scan.pdf

Chunking comes next. Docling’s documentation describes two chunkers. HierarchicalChunker emits roughly one chunk per document element and attaches headings and captions as metadata; by default it merges only list items. HybridChunker starts from that and applies tokenizer-aware refinement in two passes: split chunks that exceed the token limit, then merge undersized neighbours that share the same headings and captions (controlled by merge_peers, default true). For tables that span chunks, repeat_table_header (default true) repeats the header row so every chunk stays interpretable.

from docling.chunking import HybridChunker

# Tokenizer should match your embedding model. Constructor arguments
# differ between versions: check the hybrid chunking example notebook.
chunker = HybridChunker()

records = []
for chunk in chunker.chunk(dl_doc=doc):
    text = chunker.contextualize(chunk=chunk)   # headings + body
    page_nos = sorted({p.page_no
                       for item in chunk.meta.doc_items
                       for p in item.prov})
    records.append({"text": text, "pages": page_nos})

contextualize is the detail that pays for itself. It prepends the heading path, so a chunk that says “Q3: 4.2” is embedded as “Revenue by region > Q3: 4.2”. Without it, the embedding cannot tell which table the number belongs to.

Figure 4: PDF parsing for RAG. Heading-aware chunks keep page and bounding-box metadata, so the final answer can cite its source region.

Figure 4 shows why provenance should travel with each chunk. After retrieval and reranking (see our reranker benchmark for how rerankers change precision), the generator can cite “page 14, table 3” and the interface can draw the box. Users trust systems that show their work, and auditors require it.

Calling a VLM service for the hard pages

For pages the pipeline handles badly, call an OpenAI-compatible endpoint, which is how olmOCR’s --server mode and most vLLM deployments work. The sketch below is deliberately generic; endpoint, model name and prompt are yours to set.

import base64, requests

def page_to_markdown(png_bytes, endpoint, model):
    b64 = base64.b64encode(png_bytes).decode()
    body = {
        "model": model,
        "temperature": 0.0,
        "max_tokens": 4096,
        "messages": [{"role": "user", "content": [
            {"type": "text",
             "text": "Transcribe this page to Markdown. Use HTML for tables. "
                     "Do not add or omit text."},
            {"type": "image_url",
             "image_url": {"url": f"data:image/png;base64,{b64}"}},
        ]}],
    }
    r = requests.post(endpoint, json=body, timeout=120)
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"]

Temperature zero reduces variance but does not remove the repetition-loop risk, which is why the validation gate in Figure 2 exists.

Tables, Formulas, Handwriting and Scans

The categories where families diverge most are worth separating, because aggregate scores hide them.

Tables are the highest-value and highest-risk element. A pipeline with a dedicated structure model (TableFormer in Docling) predicts a grid of rows, columns and spans, then fills cells from the text layer or OCR. Cell content is anchored to pixels, so numbers are rarely invented. A VLM emits the table as HTML or Markdown tokens. It copes better with borderless and rotated tables but may misalign columns when a row is long. Mistral’s announcement highlights HTML output with colspan and rowspan precisely because Markdown pipes cannot express merged cells. In the olmOCR-Bench table column above, scores range from about 61 to 88, a wider spread than almost any other category: tables are where you should test your own documents first.

Formulas need LaTeX output, and the metric (CDM) renders both prediction and reference before comparing, which avoids penalising equivalent LaTeX spellings. Modern small VLMs score above 97 CDM on the OmniDocBench leaderboard. If you work in finance or operations rather than science, you can ignore this and save the compute.

Handwriting remains the weakest area for everything. Mistral lists handwriting among the strengths of OCR 3, while the olmOCR-Bench “old scans” category, which includes degraded and handwritten material, tops out near 50 for the best entry. I did not find an independent, recent handwriting benchmark that I could verify, so treat any handwriting claim as unproven for your handwriting.

Scans punish stage-wise pipelines because a small skew or noise level shifts the layout model’s boxes. VLMs tolerate this better, but have a different weakness: low-resolution input is where they hallucinate. If you control capture, scan at 300 dpi or better; no model recovers information the scanner never recorded.

Born-digital PDFs are the case people forget. If a PDF has an embedded text layer, extracting it is faster, cheaper and more exact than any OCR. The real work there is layout and reading order, which is exactly what Docling’s PDF backend does. A router that sends born-digital files to the fast path and only scans to OCR can cut compute drastically on mixed corpora.

Cost and Throughput: A Model You Can Adapt

Published prices anchor the discussion; the rest depends on your hardware and utilisation. These are the verified inputs:

Mistral OCR 3: 2 USD per 1,000 pages, or 1 USD with the 50% Batch API discount; annotations (structured extraction) are 3 USD per 1,000 pages, according to the launch post.
olmOCR paper: about 176 USD per million pages self-hosted, versus over 6,240 USD per million for GPT-4o on the same task. The current README says “less than $200 USD per million pages”. Both are the authors’ figures, tied to GPU pricing at the time.
DeepSeek-OCR: the README quotes about 2,500 tokens per second concurrency for PDF processing on an A100-40G, and the paper claims over 200,000 pages per day on one A100-40G for training-data generation. Those are vendor claims.

Converting to a common unit for a rough comparison, illustrative only:

Option	Cost per 1,000 pages	Source or basis
Mistral OCR 3, standard	2.00 USD	published price
Mistral OCR 3, batch	1.00 USD	published price
olmOCR self-hosted	about 0.18 to 0.20 USD	Ai2 figures, GPU cost dependent
Frontier general VLM via API	6 USD or more	olmOCR paper’s GPT-4o estimate, now dated
Pipeline on CPU or modest GPU	varies, mostly amortised hardware	your own measurement

The self-hosted number deserves scepticism. It assumes a saturated GPU. If your traffic is 5,000 pages a day, an idle 40 GB GPU rented at a few dollars an hour costs far more per page than the batch API, and Mistral’s batch price of 1 USD per 1,000 pages means 5,000 pages cost 5 USD. Break-even for self-hosting arrives only when you are processing enough pages to keep the GPU busy most of the day, or when privacy rules forbid sending documents out. A useful rule: compute your monthly page volume, multiply by the API price, and compare with the monthly cost of the smallest GPU instance that meets your latency need. Include engineering time for operating vLLM, handling retries, and upgrading models.

Throughput has a second dimension, which is latency per document. Pipelines parallelise across pages on CPU; VLMs are bound by autoregressive decoding of the output text, so a dense page of 2,000 output tokens takes seconds even on fast hardware, regardless of how quickly the encoder runs. Batching many pages concurrently on one server is how the published cost figures are achieved, which makes them throughput numbers, not interactive-latency numbers.

A Routing Design That Works in Practice

Figure 3: A routing design for document AI OCR. Cheap paths handle easy documents, a quality gate catches failures, and only the residue reaches a hosted API.

Figure 3 encodes a pattern I recommend for mixed corpora. First check whether the file has a usable text layer. If it does, use the layout parser fast path. If it is a scan, estimate layout complexity (single column, text only versus multi-column with tables) and choose classical OCR with a layout model for the simple case or a VLM for the complex one. Every path ends at a quality gate, and failures escalate to the best hosted option.

The quality gate is the part teams skip, and it is where most of the value lives. Cheap, model-free checks catch a large share of failures:

Length sanity. Output characters per page should fall in a plausible range for the document type; a tenth of the expected length suggests a skipped region, three times suggests a loop.
Repetition detection. A compression ratio test (zlib on the output) flags looping text, which compresses far better than prose.
Character-class checks. A page of Latin text that contains CJK or box-drawing characters is wrong.
Table checksum. For financial tables, rows should sum to totals. This is a domain check, and it is the single strongest hallucination detector for numeric data.
Cross-engine agreement. Run two cheap engines and compare; disagreement on a number flags it for review.

A gate with these five checks costs almost nothing and converts silent errors into routed ones. The same discipline applies to downstream extraction; our post on structured output benchmarks for JSON mode shows how schema validation catches model errors in the same spirit.

Trade-offs, Gotchas, and What Goes Wrong

Hallucination is a new class of OCR error. A classical engine garbles characters visibly. A generative model can complete a sentence it cannot read. Small-model proponents sometimes say hallucination risk is lower because the model is constrained to transcribe, but IBM’s own model card calls whether smaller models hallucinate more an open question. Do not assume. Measure on your documents, with numbers weighted by business impact.

Repetition loops and truncation. Long tables and repeated rows can send decoding into a loop that fills the token budget. Always set max_tokens, detect repetition, and retry with a different decoding setting or an alternative engine.

Silent skips. A VLM may drop a footnote or a small-print paragraph without any signal. The olmOCR-Bench categories “headers and footers” and “long tiny text” exist because both are common failure points: scores for the systems in the table above range from roughly 32 to 97 on the former.

Reading order across complex layouts. Newspapers, brochures and forms with side panels are ambiguous even for humans. A pipeline reads boxes in a heuristic order; a VLM decides implicitly. Neither is reliable, so test on your own layouts.

Licence and hosting. Docling is MIT, olmOCR and Granite-Docling are Apache 2.0, DeepSeek-OCR is MIT. Model weights can carry separate terms from code, as the Docling README states: individual models are governed by their own licences. Read each model’s card before shipping. Mistral OCR is a hosted API, with self-hosting offered to organisations with strict data privacy needs according to the launch post.

Benchmark overfit. Vendors tune to public benchmarks, and the benchmark authors are sometimes competitors. A two-point gap is noise. A fifteen-point gap in one category (tables, old scans) is a signal worth testing.

Version drift. Parsers change output formatting between releases. Pin versions, store the parser version with every indexed chunk, and plan for re-indexing, because a new table serialisation can silently shift your embeddings.

Data protection. Sending contracts, medical records or customer IDs to a hosted API creates a compliance question before it creates a technical one. The privacy argument for local pipelines is often stronger than the cost argument.

Evaluating on Your Own Documents

A defensible evaluation takes a day, not a quarter, and it will outperform any leaderboard in predicting production quality.

Sample 100 to 200 pages stratified by type: clean digital, scans, tables, forms, multi-column, and your worst offenders. Include documents that have already caused support tickets.
Create ground truth for what matters. You do not need full transcription. Mark the 10 to 20 facts per page the application will actually use: totals, dates, party names, part numbers, table cells.
Score at the fact level. For each system, count exact matches, near matches and wrong values. Separate “missing” from “wrong”; missing is recoverable by review, wrong is not.
Add a detection score. Of the wrong values, how many did your quality gate flag? A system with a 3% error rate and 90% detection is safer than a system with a 1% error rate and no detection.
Measure cost and latency under the real concurrency, not a single warm request.
Repeat quarterly. Models improve fast, and routes that were right in January may be wrong by autumn.

For text-heavy corpora, normalised edit distance over the full page is a reasonable proxy and is what OmniDocBench uses. For tables, use TEDS or cell-level F1. For formulas, CDM or a rendered comparison. For retrieval impact, which is the thing you finally care about, measure answer accuracy and citation correctness on a set of questions, since parsing improvements that do not move that number are not worth their cost.

Decision Matrix

Situation	Best fit	Why	Watch out for
Born-digital PDFs, structure and provenance needed	Docling or similar layout parser	Reads text layer, layout aware, bounding boxes, MIT	OCR not needed; avoid running it
Large scanned archive, privacy required, GPUs available	olmOCR or comparable open VLM on vLLM	Roughly 0.2 USD per 1,000 pages at saturation per Ai2	Hallucination, loops, GPU ops
Low volume, minimal ops, tables and forms	Mistral OCR API	1 to 2 USD per 1,000 pages, HTML tables	Data leaves your network; private benchmarks
Mixed corpus at scale	Routed hybrid with quality gate	Cheap path for easy pages, escalation for hard	More moving parts to monitor
Edge or CPU-only deployment	Docling default pipeline or Granite-Docling on a small GPU	Small models, 258M-parameter VLM exists	Accuracy limits on messy scans
Heavy equations and academic PDFs	PaddleOCR-VL family or MinerU on OmniDocBench-style data	Top formula CDM on public leaderboard	Check licence and your own subject area
Visual questions over charts and layouts	Page-image retrieval (ColPali style)	Skips parsing entirely	No clean text for audit

Practical Recommendations

Start with the corpus, not the model. Classify your documents into digital, clean scan and degraded scan, estimate the share of each, and note which contain tables whose numbers drive decisions. That alone usually removes half of the options.

For most teams the best first build is a Docling-based pipeline with the default layout and table models, lossless JSON stored alongside Markdown, and the hybrid chunker feeding your embedder. It is open, runs locally, preserves provenance and is easy to debug. Add a VLM only where measurements show the pipeline losing, and keep the VLM behind the same DoclingDocument interface so downstream code does not change.

Use a hosted API as the escalation tier rather than the default if volume is high, and as the default if volume is low and privacy permits. Whatever you choose, build the quality gate on day one. It is cheap, and it turns the failure mode that matters (confident wrong values) into a review queue.

A short checklist:

Keep the text layer for born-digital PDFs; run OCR only on pages lacking one.
Store page numbers and bounding boxes with every chunk.
Contextualise chunks with their heading path before embedding.
Pin parser and model versions; record them in chunk metadata.
Set token limits and detect repetition on every VLM call.
Validate numeric tables with domain checks such as row sums.
Evaluate on your own stratified sample before trusting any leaderboard.
Re-run the evaluation each quarter and when you change models.

For wider context on how vision and language models fuse internally, see our overview of multimodal AI architecture.

Frequently Asked Questions

What is the difference between OCR and document AI?

OCR converts image pixels into characters. Document AI adds understanding of structure: it identifies titles, paragraphs, tables, figures and formulas, restores reading order, and outputs structured formats such as Markdown, HTML or JSON with positions. Modern document AI may combine a classical OCR engine with layout and table models, or use one vision-language model that does all of it in a single pass. The extra structure is what makes the output usable for retrieval, extraction and audit.

Is Docling better than olmOCR?

They solve overlapping problems differently, so neither is simply better. Docling is a pipeline and document model that preserves layout and provenance, runs locally under an MIT licence, and can also drive a small VLM (Granite-Docling). olmOCR is a 7B-parameter VLM that transcribes a page end to end, and scores about 82 on its own benchmark. Choose Docling for structured, auditable output, and olmOCR for difficult scans where pipelines fail.

Do vision-language models hallucinate when doing OCR?

Yes, it is a real risk. Because a VLM generates text rather than classifying characters, it can fill in unreadable content, skip passages or loop on repeated rows. The risk rises on low-resolution or degraded input. Mitigate with token limits, repetition checks, domain validation such as table totals, and escalation to a second engine. Whether smaller models hallucinate more is described as unresolved on IBM’s Granite-Docling model card, so measure on your own data.

How much does OCR cost per 1,000 pages in 2026?

Published prices put Mistral OCR 3 at 2 USD per 1,000 pages, or 1 USD through its batch discount. Ai2 reports self-hosting olmOCR for under 200 USD per million pages, about 0.20 USD per 1,000, at high utilisation. Real costs depend on GPU utilisation, engineering time and volume. At low volume an API is almost always cheaper than an idle GPU, and these figures are vendor-reported rather than independently audited.

What is the best PDF parsing approach for RAG?

Preserve structure and provenance. Parse with a layout-aware tool, keep the text layer on born-digital PDFs, serialise tables as HTML or Markdown that keeps headers, then chunk by document structure rather than fixed character counts. Prepend the heading path before embedding and store page and bounding-box metadata for citations. Docling’s hybrid chunker does heading-aware splitting and merging against a tokenizer limit. Evaluate end to end on answer accuracy, not parsing scores alone.

Which benchmark should I trust for OCR accuracy?

None alone. OmniDocBench offers broad, structured coverage across 1,651 pages and several metrics. olmOCR-Bench uses pass/fail unit tests and exposes weaknesses such as old scans. Both are produced or run by parties with interests, scores within about a point are within noise, and neither covers your scanners or languages. Use them to shortlist, then decide using a stratified sample of your own documents.

Document AI in 2026: OCR Pipelines vs Vision-Language Models (Docling, olmOCR, Mistral OCR)