Matryoshka Embeddings: The Adaptive-Dimension Retrieval Architecture

Most vector search stacks pick one embedding size at design time and live with the trade-off forever. Matryoshka embeddings break that constraint: a single vector is trained so that its first k coordinates are, on their own, a fully usable lower-dimensional embedding — nested like a set of wooden dolls, each complete by itself. A 3072-dimensional vector trained with Matryoshka Representation Learning (MRL) can be sliced down to 256 or even 64 dimensions, re-normalized, and dropped straight into an approximate nearest-neighbor index — no re-embedding pass, no second model, no separate training run. That turns a fixed storage-versus-accuracy decision into a runtime knob, which is exactly why adaptive, coarse-to-fine retrieval has become the default pattern for large-scale RAG and search systems built since 2024.

What this covers: the MRL training objective and why it front-loads information into leading coordinates, the two-stage coarse-to-fine retrieval architecture built on top of it, worked storage and RAM math at scale, and the specific failure modes — recall cliffs, truncating non-MRL models, shortlist miscalibration — that break this pattern in production.

Context and Background

Dense retrieval has always fixed its embedding dimension at training time. Classic bi-encoders — BERT-based sentence encoders, OpenAI’s text-embedding-ada-002, early Cohere and Sentence-Transformers models — shipped one output size, typically 384, 768, or 1536 dimensions, baked into the model card. If a team needed a smaller footprint for a mobile index or a cheaper ANN cluster, the options were unattractive: retrain a smaller model from scratch, or bolt on a post-hoc reduction like PCA or a learned linear projection. Both are lossy in ways that are hard to predict, and both still require rebuilding the index from a second, separately produced set of vectors.

Matryoshka Representation Learning changed the calculus. Kusupati et al. introduced MRL in “Matryoshka Representation Learning” (NeurIPS 2022), originally demonstrating it on ImageNet classification and retrieval embeddings, showing that a single ResNet or ViT representation could be trained to remain accurate when truncated to a fraction of its trained width. The technique migrated into text embeddings over the following two years. By 2024, OpenAI’s text-embedding-3-small and text-embedding-3-large shipped with a dimensions parameter in the embeddings API, letting callers request a shorter vector directly rather than truncating client-side. Nomic’s nomic-embed-text-v1.5 documented explicit Matryoshka training, and subsequent open releases in the BGE, GTE, and Jina families followed with MRL-aware variants or fine-tunes. Not every embedding model on the market is Matryoshka-trained, though, and confusing “supports a dimensions parameter” with “was actually trained to preserve accuracy at that dimension” is one of the most common mistakes teams make when picking a model — a distinction covered in more depth in our embedding model benchmark comparing OpenAI, Cohere, Voyage, and BGE.

The idea has intellectual roots even further back than 2022. Nested dropout and stochastic-depth training — techniques that randomly ordered which units in a network were “essential” versus “optional” during training — share the same underlying intuition: representations can be trained to degrade gracefully rather than catastrophically when part of them is removed. What MRL contributed was a clean, deterministic version of that idea aimed specifically at embedding vectors used for retrieval and classification, with an explicit multi-granularity loss rather than a stochastic masking scheme, which is why it produces predictable nested prefixes instead of a randomly-degrading representation.

Adoption accelerated once major embedding providers started shipping the training approach rather than leaving teams to discover it independently. By 2025, Matryoshka embeddings support had become close to table stakes for any newly released general-purpose text embedding model, alongside longer context windows and better multilingual coverage, because vendors found it directly addressed the single most common complaint from teams running retrieval at scale: index cost. That shift matters for anyone architecting a retrieval system today — it’s no longer a research curiosity you have to fine-tune yourself, but a property you should actively check for and select on when choosing a base model.

The practical draw is simple: storage and compute for vector search scale linearly with dimension, and dimension has historically been a one-time, whole-system decision. MRL turns it into a per-query, per-tier decision instead — which is the architectural pattern this post walks through end to end.

The Core Architecture: Nested Dimensions and Adaptive Retrieval

Matryoshka embeddings work by training one encoder so that truncating its output vector at any of several nested dimensions — say 3072, 1024, 256, and 64 — still yields a valid, usable embedding for similarity search. Adaptive retrieval exploits this directly: shortlist candidates cheaply with a truncated vector, then re-rank the shortlist using the full-dimension vector for final precision.

Figure 1: A single Matryoshka-trained embedding nests multiple usable dimensions; adaptive retrieval truncates for a cheap shortlist pass, then re-ranks with the full vector.

The diagram captures two properties working together: a training-time guarantee (nesting) and an inference-time pattern built on top of it (coarse-to-fine search). Neither idea is new in isolation — dimensionality reduction and multi-stage retrieval both predate Matryoshka embeddings by a decade — but training the model to make low-dimensional prefixes reliably accurate is what makes combining them cheap and safe instead of a lossy hack layered onto an unrelated model.

Why Truncation Doesn’t Break Similarity

An ordinary encoder, trained with a single loss at its full output width, spreads the information needed to reconstruct semantic similarity across all of its coordinates in whatever pattern gradient descent happens to find. There is no reason coordinates 1–256 of a standard 1536-dimensional embedding would be any more informative than coordinates 1281–1536; slice off the tail and you are discarding a essentially random subset of the model’s signal, which is why naive truncation of a non-Matryoshka model degrades quality unpredictably.

MRL changes the objective, not the architecture. During training, the model computes a loss not just on the full-width embedding but on several nested prefixes of it simultaneously, then sums (optionally weights) those losses before backpropagating. Because the leading 64 coordinates are directly optimized to work as a standalone 64-dimensional embedding — not just as part of a larger vector — information is pushed toward the front of the vector in decreasing order of importance, similar in spirit to how PCA orders principal components by explained variance, except learned end-to-end through the same contrastive or classification objective the full model already uses. The first k coordinates of an MRL vector are a coherent, self-contained low-rank summary, not an arbitrary slice.

The contrast matters concretely. Take two 1024-dimensional embeddings of the same document, one from a standard encoder and one from an MRL-trained encoder, and slice both down to their first 128 coordinates. The standard slice behaves like a random 128-dimensional projection of the full vector — it retains some signal simply because embeddings are somewhat redundant, but there’s no guarantee any particular 128 of the 1024 coordinates were the ones worth keeping, and different training runs of the same architecture would front-load information differently or not at all. The MRL slice, by contrast, was explicitly evaluated and optimized as a 128-dimensional embedding during every training step, so its behavior at that width is a designed property of the model, not an accident of which coordinates the optimizer happened to favor.

Where the Two Stages Live in a Real Stack

In a production system, the split between the two stages maps cleanly onto the existing write path and query path. At ingestion time, the encoder runs once per document and produces the single full-width vector — this is the only place the model is invoked, and it happens exactly as often as it would in a fixed-dimension system. That vector is then written twice, at different fidelities: a truncated, renormalized copy goes into the low-dimension ANN index that serves the shortlist stage, and either the full vector or a compressed version of it goes into a secondary store — a key-value store, a columnar table, or a second, smaller ANN index — that serves the rerank stage. At query time, the same encoder call produces one query embedding, which is likewise truncated for the shortlist search and used at full width for the rerank. Nothing about this requires touching the encoder more than once per document or per query; the two stages differ only in which stored representation they read.

The Coarse-to-Fine Retrieval Pattern

The retrieval architecture built on this property runs in two stages against a single stored embedding per item. First, an approximate nearest-neighbor index — HNSW, IVF-PQ, or similar — is built over the truncated, low-dimensional slice of every vector (128–256 dimensions is a common working range). A query is embedded once, truncated the same way, and searched against that index to produce a shortlist, typically 100–2000 candidates depending on corpus size and target recall. Second, the shortlist is re-scored using the full-dimension vectors — either the untruncated query and candidate embeddings pulled from a secondary store, or (for smaller shortlists) a brute-force cosine computation done in memory — and the top-k after re-ranking is what gets returned to the caller.

The economics work because the expensive step, a full nearest-neighbor search over the entire corpus, only ever runs at low dimension, and the accurate step, full-dimension comparison, only ever runs over a small shortlist rather than the whole collection. Both stages consume the same embedding produced by a single encoder call at write time; there’s no second model to host, version, or keep in sync, and no re-embedding job when a team decides to change the shortlist dimension.

Normalization After Truncation

One detail is easy to miss and quietly wrong if skipped: cosine similarity assumes unit-norm vectors, and truncating a vector changes its L2 norm because you’ve discarded some of the magnitude that contributed to it. A truncated-but-not-renormalized vector will produce systematically distorted similarity scores relative to the original full vector’s geometry. The correct operation is truncate-then-normalize, applied consistently to both queries and stored vectors. Some client libraries and APIs — OpenAI’s dimensions parameter among them — perform this renormalization server-side automatically; Sentence-Transformers’ truncate_dim argument does the same when paired with normalize_embeddings=True. If you are truncating vectors manually before writing them into an index, renormalization is not optional. Because renormalized truncated vectors remain unit-norm, cosine similarity and dot-product (MIPS) search remain equivalent at every truncation level, which is what lets the same ANN infrastructure serve both stages of the pipeline without special-casing the distance function.

Deeper Walk-through: MRL Training and the Coarse-to-Fine Pipeline at Scale

Understanding why the pattern is worth adopting requires looking at both sides: how the loss is actually constructed during training, and what it buys you in storage and RAM once you deploy it at real corpus sizes.

Figure 2: MRL trains one encoder against losses computed at several nested dimensions simultaneously, then sums them into a single backward pass.

Choosing granularities and weighting the loss

MRL does not train a separate head per dimension. A single forward pass produces one full-width embedding; a small, fixed list of nested prefix lengths — commonly a geometric-ish progression such as 64, 128, 256, 512, 1024, 2048, and the full width — is sliced from that same vector, and the task loss (contrastive loss for retrieval, cross-entropy for classification) is computed independently at each slice. Those per-granularity losses are then summed into a single scalar before the backward pass, so one gradient update improves accuracy at every nested dimension at once. The original paper found that equal weighting across granularities was a strong, simple default; relative importance weighting is possible but not required to get the core benefit. The computational overhead versus standard training is small — a handful of extra matrix slices and loss evaluations per batch, not extra forward passes through the encoder — which is why MRL-trained models cost roughly the same to train as their fixed-dimension counterparts.

This is also why MRL generalizes cleanly from classification to retrieval: the mechanism only cares that the task loss is differentiable and computable on a vector slice, so the same recipe applies to a softmax classification head or an in-batch contrastive retrieval loss without structural changes. Adaptive classification (route easy examples to a cheap low-dimensional classifier, hard ones to the full model) and adaptive retrieval (coarse shortlist, fine re-rank) are the same underlying idea applied to different downstream tasks.

There’s a real design trade-off in how many granularities to include in the nested list. Too few — say, only the full width and one small dimension — leaves large gaps where intermediate truncations behave unpredictably, since the model was never explicitly optimized at those widths, even though they typically still work reasonably well due to the ordering property. Too many granularities adds training overhead for diminishing returns and can slightly dilute the full-dimension performance versus a model trained with a single loss, since the objective now has to serve several masters at once. In practice, most released MRL text embedding models settle on somewhere between four and eight granularities spanning roughly two orders of magnitude, which balances flexibility for downstream users against training complexity.

It’s also worth being precise about what MRL does not do: it does not make an arbitrary intermediate dimension free to compute cheaper embeddings for — the encoder still does the same amount of work per document regardless of what dimension you eventually truncate to, since the full vector is always produced first. The savings are entirely on the storage, indexing, and search side, not on the embedding-generation side. Teams sometimes conflate “smaller embeddings” with “cheaper to generate,” and MRL only delivers the former; if inference cost of the encoder itself is the bottleneck, a genuinely smaller model, not a truncated vector from a larger one, is the right lever.

Storage and RAM math at production scale

Figure 3: The low-dimension index carries the RAM cost of ANN search; full vectors are fetched only for the shortlist that survives it.

Take a corpus of 100 million vectors at 3072 dimensions, stored as float32 (4 bytes per value):

Dimension	Bytes / vector	Raw storage, 100M vectors	Typical HNSW RAM (≈1.5–2× raw)	Role in the pipeline
3072 (full)	12,288 B	≈1.14 TB	≈1.7–2.3 TB	Rerank only, fetched for shortlist
1024	4,096 B	≈381 GB	≈570–760 GB	Mid-tier shortlist, larger corpora
256	1,024 B	≈95 GB	≈143–190 GB	Primary shortlist tier
64	256 B	≈24 GB	≈36–48 GB	Aggressive coarse pass, high-recall tolerance needed

The full-dimension index alone needs on the order of 2 TB of RAM to serve with HNSW-style graph overhead at this corpus size — multi-node territory. The 256-dimension index for the same 100M vectors fits comfortably on a single large memory-optimized instance, roughly a 10–12x reduction in index RAM for the shortlist stage, while the full vectors can live in cheaper, non-ANN storage (object storage, a key-value store, or a columnar store) and be fetched only for the few hundred to few thousand candidates that survive the coarse pass. Layering standard compression on top compounds the savings further: int8 scalar quantization typically shrinks vectors roughly 4x with a small, well-characterized recall cost, and binary quantization (1 bit per dimension) can shrink them another order of magnitude beyond that, though binary-quantized shortlists need a wider net and a mandatory full-precision rerank stage to recover competitive recall — Matryoshka truncation and PQ/binary quantization are complementary, not competing, techniques, and production systems commonly stack a truncated dimension with int8 or binary compression on top of it.

The recall behavior across this table is intentionally described qualitatively rather than with fabricated precision: the actual recall-versus-dimension curve for a given model, domain, and query distribution has to be measured on your own held-out evaluation set. What is consistent across MRL-trained models in published evaluations is the shape of the curve — recall stays close to the full-dimension baseline down to a moderate fraction of the trained width, then falls off increasingly steeply as dimension keeps shrinking. Where that knee sits is model- and domain-specific, which is exactly why the shortlist size and truncation dimension need calibration rather than a default guess, covered in the trade-offs section below.

Latency follows a similar shape to storage, for a related reason: HNSW and IVF search cost is dominated by the number of distance computations times the cost per distance computation, and a distance computation between two vectors is roughly linear in dimension. Cutting the shortlist-stage dimension from 3072 to 256 doesn’t just shrink the index — it also shrinks the per-comparison cost by roughly the same 12x factor, on top of any reduction in the number of comparisons the graph traversal needs to make at a smaller working set size. The rerank stage doesn’t erase this win as long as the shortlist stays small relative to the corpus: reranking 500 full-dimension vectors is a bounded, constant-size cost regardless of whether the underlying corpus is 1 million or 1 billion items, which is precisely the property that makes the two-stage pattern scale where a single full-dimension ANN search does not.

A decision matrix for choosing the shortlist tier

Corpus size	Recommended shortlist dim	Shortlist size (top-N)	Rerank dim	When to use
<1M vectors	Full dim	N/A (single stage)	N/A	Truncation overhead isn’t worth it below this scale
1M–20M	256–512	200–500	Full	Standard RAG corpus, single-node ANN
20M–200M	128–256	500–2000	Full or 1024	Multi-tenant search, cost-sensitive infra
200M+	64–128	2000–5000	512–1024, then full on final top-k	Web-scale catalogs; consider a three-stage cascade

At the largest end of that range, a two-stage pipeline can be extended into a three-stage cascade rather than jumping straight from a very low dimension to full precision: an aggressive 64-dimensional pass narrows a billion-item corpus down to a few thousand candidates, a mid-tier 512-dimensional pass narrows that down to a few hundred, and only the final few hundred are compared at full width. Each additional stage trades a small amount of extra pipeline complexity for a further reduction in how much full-precision computation the system ever has to do, which matters more as corpus size grows, since the cost of the coarse stage is the only one that scales with total corpus size — every stage after it operates on a bounded, pre-filtered candidate set.

Trade-offs, Gotchas, and What Goes Wrong

The pattern’s biggest risk is that its most common failure is silent. Truncating an embedding model that was never trained to produce Matryoshka embeddings still produces a vector, and cosine similarity still returns a number — nothing throws an error. But because a non-Matryoshka model spreads information arbitrarily across its coordinates, the truncated vector is not a coherent low-dimensional summary, and recall against true nearest neighbors can degrade sharply with no warning signal in the code path. The only reliable check is an explicit recall evaluation comparing truncated-vector retrieval against full-dimension ground truth on a labeled or self-supervised eval set, before the truncated index goes anywhere near production traffic.

Figure 4: Truncation is safe only after two checks — does the model actually use Matryoshka embeddings, and does the target dimension sit above the model’s recall cliff.

Even with a genuinely MRL-trained model, there is a recall cliff: recall tracks the full-dimension baseline closely down to some fraction of the trained width, then drops steeply below it. That knee point is model- and domain-specific and has to be measured, not assumed from a vendor’s marketing dimension list. Re-ranking cost is the second lever to get wrong in both directions — too small a shortlist and the true top-k results never make it into the candidate set for the rerank stage to find, silently capping recall regardless of how good the full-dimension reranker is; too large a shortlist and the fetch-and-rerank stage erodes the latency and cost savings the coarse pass was supposed to buy. Mixing truncation dimensions inside a single ANN index is also a common misconfiguration: HNSW and IVF both assume one fixed dimensionality per index, so vectors truncated to different lengths cannot coexist in the same index or be compared directly against each other — every item in a given index tier must be truncated to the same dimension.

A few more edge cases worth planning for explicitly. High-stakes ranking — legal discovery, medical literature retrieval, security-sensitive search — often still warrants full-dimension search throughout, or at minimum a much larger, more conservative shortlist, because the cost of a missed relevant document outweighs the infrastructure savings. Domain and language drift matter too: MRL’s nesting behavior is learned from the training distribution, and a model fine-tuned or evaluated only on English web text may not preserve the same low-dimension recall guarantees on a specialized technical corpus or a low-resource language — re-validate the recall curve whenever the deployment domain diverges meaningfully from the model’s training data, not just once at launch.

Two operational gotchas round out the list. First, changing the shortlist-tier truncation dimension after launch — because a recall audit found the current setting underperforming, say — is not a config flag flip; it requires rebuilding the entire low-dimension ANN index from the already-stored full vectors, which is cheap relative to re-embedding the corpus but not instantaneous, and needs a migration plan (dual-write, shadow index, or a maintenance window) rather than an in-place change. Second, recall drift over time is easy to miss because nothing alerts on it by default: as a corpus grows or its topic distribution shifts, the shortlist size and dimension that were calibrated at launch can quietly become insufficient months later. Treat the recall evaluation as a recurring check, not a one-time gate, the same way you’d monitor any other model-quality metric in production.

Practical Recommendations

Treat the truncation dimension as a tuned hyperparameter, not a fixed architectural constant. Start by confirming — from the model card or a direct citation, not an assumption — that the embedding model you’re using was actually trained with MRL; a dimensions parameter in an API is a strong signal but not definitive proof on its own. Build a small labeled or self-supervised recall evaluation set specific to your domain before choosing a shortlist dimension, sweep a handful of candidate dimensions and shortlist sizes against it, and pick the smallest dimension that keeps recall within an acceptable tolerance of the full-dimension baseline.

The self-supervised version of that evaluation set is worth calling out because it removes the excuse of “we don’t have labeled relevance data yet.” Take a sample of real queries, run full-dimension exact nearest-neighbor search to establish ground truth, then measure what fraction of that ground-truth top-k survives into the truncated-dimension shortlist at various shortlist sizes. This requires no human labeling at all, can be automated as a recurring job, and directly answers the two questions that matter — which dimension to truncate to, and how large the shortlist needs to be — using the exact corpus and query distribution the system will actually see in production, rather than a published benchmark that may not resemble your domain. If you’re deciding between a general-purpose vector database and a Postgres extension for the index layer, the storage-tier math above is one of the deciding factors worth reading alongside our pgvector vs. dedicated vector database comparison — a truncated-dimension shortlist tier is often small enough to fit comfortably inside pgvector even when the full-dimension store needs a purpose-built engine. Where relevance quality matters more than raw dense recall, this pattern also composes with sparse retrieval; see our hybrid dense-sparse fusion architecture with RRF for combining a Matryoshka-truncated dense shortlist with a lexical (BM25/SPLADE) signal before the final rerank.

Checklist before shipping an adaptive retrieval pipeline:

Confirm the embedding model genuinely uses Matryoshka embeddings, with a citation, not just a documented dimensions parameter.
Re-normalize every vector after truncation, both queries and stored candidates, before computing cosine or dot-product similarity.
Measure the recall-versus-dimension curve on your own domain data; don’t reuse a vendor benchmark’s knee point.
Calibrate shortlist size against measured recall, then re-check it whenever corpus size or domain shifts meaningfully.
Keep exactly one truncation dimension per ANN index; never mix truncated and full vectors in the same index.
Decide explicitly, per use case, whether the final rerank needs full-dimension vectors or can stop at an intermediate tier.
Re-validate periodically for domain or language drift, not just at initial launch.

Frequently Asked Questions

What is a Matryoshka embedding?

A Matryoshka embedding is a vector produced by a model trained with Matryoshka Representation Learning (MRL), such that its first k coordinates — for several nested values of k — are each independently usable as a lower-dimensional embedding. Truncating the vector and re-normalizing it produces a smaller embedding without re-running the model, unlike standard fixed-dimension embeddings where any subset of coordinates is effectively random and unreliable for similarity search.

How is MRL training different from standard contrastive training?

Standard contrastive or classification training computes one loss at the model’s full output width. MRL computes that same loss at several nested prefix lengths of the same output vector — for example 64, 256, 1024, and full width — sums them into one scalar, and backpropagates once. The extra cost is a few matrix slices and loss evaluations per batch, not additional forward passes, so MRL training is only marginally more expensive than standard training.

Can I truncate any embedding model like a Matryoshka embedding?

No. Truncating a model that wasn’t trained with MRL still produces a vector and a similarity score, but the discarded coordinates weren’t optimized to be unnecessary — the remaining coordinates are not a coherent low-dimensional summary. Recall degrades unpredictably and silently, with no error to signal the failure. Only truncate models explicitly documented as Matryoshka-trained, and verify the recall curve yourself.

What dimension should I use for the shortlist stage?

There’s no universal number; it depends on the model’s recall cliff, corpus size, and latency budget. A common starting range is 128–256 dimensions for corpora up to roughly 200 million vectors, widening the shortlist size (not just the dimension) if recall measured against a full-dimension baseline falls short. Always validate with your own eval set rather than copying a vendor’s suggested default.

Do Matryoshka embeddings work with binary or int8 quantization?

Yes, and the two techniques are complementary. MRL reduces the number of dimensions stored; scalar (int8) or binary quantization reduces the bits per dimension. Stacking both — for example a 256-dimension MRL truncation stored as int8 — compounds the storage savings, though more aggressive binary quantization typically needs a wider candidate shortlist and a mandatory full-precision rerank to hold recall steady.

Does OpenAI’s text-embedding-3 use Matryoshka Representation Learning?

OpenAI documents that text-embedding-3-small and text-embedding-3-large support a dimensions parameter that shortens the embedding while preserving its concept-representing properties, consistent with a Matryoshka-style training approach, and recommends this over naive post-hoc truncation of older models. For exact current guidance, always check OpenAI’s embeddings documentation directly, since model-specific behavior can change between releases.

Matryoshka Embeddings: Adaptive-Dimension Retrieval Architecture (2026)