Prometheus Long-Term Storage: Thanos vs Mimir vs VictoriaMetrics (2026)

A single Prometheus server is a wonderful thing until the day someone asks for last year’s p99 latency during Black Friday. Local TSDB retention is measured in weeks, the disk is a single point of failure, and the query that spans thirteen months either times out or never had the data. Every team that runs Prometheus at scale eventually shops for Prometheus long term storage, and the shortlist is almost always the same three projects: Thanos, Grafana Mimir and VictoriaMetrics.

They solve the same problem with three quite different philosophies. Thanos keeps Prometheus at the centre and bolts object storage on. Mimir turns Prometheus ingestion into a horizontally scaled, multi-tenant service. VictoriaMetrics replaces the storage engine with something denser and simpler to operate. This guide compares them at the level of write path, query path, failure modes, cardinality limits and cost, so you can pick on evidence rather than on a vendor slide.

What this covers: why Prometheus alone is not enough, the three reference architectures, remote write and ingestion, downsampling, multi-tenancy, high availability, cardinality control, illustrative cost at 10M and 100M active series, migration paths and a decision matrix.

Context and Background

Prometheus was designed as a single-node, pull-based monitoring system with a local time-series database. Its own documentation is candid about this: local storage is not clustered or replicated, so it is not arbitrarily scalable or durable in the face of drive or node outages, and the project recommends treating it as a short-term store and using the remote read and write interfaces for long-term retention (Prometheus storage documentation). The same page puts the average cost at roughly one to two bytes per sample, which is the number that makes every capacity estimate in this article possible.

Three pressures push teams past a single server. First, retention: compliance, capacity planning and year-over-year comparisons need months or years of history. Second, scale: once a server holds several million active series, memory, restart time (WAL replay) and query latency all degrade together. Third, availability: running two identical Prometheus replicas gives you redundancy for scraping but hands you duplicate series that something must deduplicate at query time. The wider monitoring picture matters here too; if you are still deciding what to collect and where, our OpenTelemetry vs Prometheus and Loki edge observability ADR frames the collection side of the problem.

The Prometheus ecosystem answered with two integration styles. The first is block shipping: the TSDB writes immutable two-hour blocks, and a helper uploads them to object storage. The second is Prometheus remote write, a protocol in which Prometheus streams samples from its write-ahead log to a remote endpoint. Thanos supports both styles, Mimir and VictoriaMetrics are built primarily around remote write, and all three speak PromQL on the read side (VictoriaMetrics with a superset called MetricsQL).

There is also a quiet fourth contender: managed services such as Grafana Cloud, Amazon Managed Service for Prometheus and Google Managed Prometheus. They mostly wrap the same ideas. We focus on the self-hosted trio because their architectures expose the trade-offs that managed offerings hide behind a bill.

Before comparing, a vocabulary check. An active series is a unique combination of metric name and label set that received a sample recently; it is the unit that drives memory. Metrics cardinality is the count of such series, and churn is how quickly old series are replaced by new ones (every Kubernetes pod restart mints new series if a pod label is present). A system can store a huge number of samples cheaply but still fall over on cardinality, because the index and the in-memory head are indexed by series, not by sample.

Three Reference Architectures

Short answer: how do Thanos, Mimir and VictoriaMetrics differ? Thanos federates many Prometheus servers and their object-storage blocks behind a global query layer. Mimir is a horizontally scalable, multi-tenant remote-write backend that stores TSDB blocks in object storage. VictoriaMetrics is a purpose-built database, in single-node or shared-nothing cluster form, with its own compressed storage format.

Figure 1: Where each system sits relative to Prometheus. Thanos pulls blocks from sidecars or accepts remote write, Mimir and VictoriaMetrics are remote-write targets, and all three feed Grafana through a PromQL-compatible query layer.

Figure 1 shows the three topologies side by side. The important thing to notice is the direction of coupling. With Thanos in sidecar mode your Prometheus servers remain the source of truth for recent data and Thanos is largely a read-time federation layer. With Mimir and VictoriaMetrics, Prometheus becomes a scraper and forwarder, and the long-term system owns ingestion, replication and querying. That single decision shapes failure modes, operations and cost more than any benchmark.

Thanos: Prometheus at the centre

Thanos is a set of components that you adopt incrementally. The sidecar runs next to each Prometheus, exposes its recent data through the Store API and uploads each completed block to object storage. The store gateway serves historical blocks from the bucket, syncing block metadata and caching index lookups. The querier is stateless: it implements PromQL on top of the Store APIs, fans out to sidecars and store gateways, and deduplicates replicas by external labels. The compactor runs against the bucket only, merging blocks, applying retention and creating downsampled data. The ruler evaluates rules against the global view. The architecture is documented in the Thanos design notes.

The receive component adds a second ingestion style for environments where Prometheus can only push, such as air-gapped or egress-only networks. It implements the remote write API, distributes series over a hashring, and writes to per-tenant TSDBs that are later uploaded as blocks. Replication is controlled by --receive.replication-factor (default 1) and a write succeeds when quorum, computed as RF divided by two plus one, is met, with the documented exception that an RF of 2 needs only one success (Thanos Receive documentation). The docs recommend the Ketama algorithm for new hashrings, because the historical default (hashmod) causes series churn and memory spikes when nodes are added.

Thanos is the least invasive option. If you already run ten Prometheus servers, adding sidecars and a querier gives you a global view without moving your write path. The price is that you keep operating all ten Prometheus servers, including their memory, their WAL and their upgrade cadence.

Grafana Mimir: Prometheus as a service

Mimir descends from Cortex and uses a read and write path split into stateless and stateful roles. Its documented components are the distributor, ingester, querier, query-frontend, query-scheduler, store-gateway and compactor, with an optional ruler, alertmanager and overrides-exporter (Mimir components).

On the write path, the distributor validates samples, applies per-tenant limits, optionally deduplicates HA pairs, and shards series across ingesters with a hash ring. Ingesters keep recent samples in memory, persist them to a write-ahead log, and by default compact in-memory series into TSDB blocks that are uploaded to object storage every two hours. In the classic architecture a series is written to three ingesters (replication factor 3) and a write succeeds on quorum, so two of three. Replicas can be spread across availability zones with zone-aware replication so that losing a zone does not lose data or interrupt ingestion (Mimir ingester documentation).

On the read path, the query-frontend splits and caches queries, the query-scheduler queues them fairly per tenant, queriers execute them against ingesters (recent data) and store-gateways (blocks in the bucket), and the compactor merges blocks in the background. Mimir 3.0, announced in November 2025, introduced an optional ingest storage architecture that puts Apache Kafka between distributors and ingesters, so reads and writes scale and fail independently (Grafana Mimir 3.0 release notes blog). The same post says the Mimir query engine, default in 3.0, reduces peak memory usage by up to 92 percent and that Grafana Labs saw up to 15 percent lower resource usage on its large internal clusters. Treat both as vendor-reported figures from their workloads, not guarantees.

VictoriaMetrics: a different storage engine

VictoriaMetrics does not reuse the Prometheus TSDB. It stores data in its own merge-tree-style format with aggressive compression, and it ships as a single binary for single-node use or as a cluster with three roles. vminsert accepts writes and spreads them over storage nodes by consistent hashing on the metric name and all labels. vmstorage holds the data. vmselect answers queries by fetching from all storage nodes. The documentation stresses that vmstorage nodes do not know about each other, communicate with each other or share data, which is what “shared-nothing” means in practice (cluster VictoriaMetrics documentation).

That design has a consequence people underestimate: there is no ring, no gossip and no object-storage dependency on the hot path. Storage nodes use local disks, and durability comes from the -replicationFactor flag on vminsert, which writes N copies to N distinct vmstorage nodes. The docs state that you need at least 2N minus 1 storage nodes to keep the factor when N minus 1 nodes are down, and that replication does not replace backups, which are taken with vmbackup from instant snapshots.

The single-node edition deserves its own mention. The project recommends single-node VictoriaMetrics for ingestion below roughly one million data points per second, and a single node at that rate covers a surprisingly large fleet. For many teams the correct answer to “which distributed system should we run” is “none yet”.

Write Path and Prometheus Remote Write in Detail

The write path is where the three systems diverge most, and where most production incidents begin. Understanding it also explains their scaling limits.

Figure 2: Write paths. Thanos Receive and Mimir shard by series hash and write to per-tenant TSDB heads backed by a WAL. VictoriaMetrics vminsert shards by consistent hash straight to vmstorage disks.

How remote write behaves

Prometheus remote write tails the WAL, batches samples into shards (parallel senders) and sends snappy-compressed protobuf over HTTP. The sending side is tunable through queue_config: capacity, max_shards, min_shards, max_samples_per_send and batch_send_deadline. When the receiver is slow or down, the queue fills, Prometheus keeps reading from the WAL up to a limit, and then drops the oldest data because the WAL is truncated every two hours. The Prometheus documentation notes that remote write adds memory overhead and recommends watching the prometheus_remote_storage_* metrics, in particular the gap between highest timestamp in and highest sent timestamp.

# prometheus.yml: one Prometheus, two remote write targets for a migration window
global:
  scrape_interval: 15s
  external_labels:
    cluster: prod-eu-1
    replica: prom-0          # drop this label at the backend for HA dedup

remote_write:
  - url: https://mimir.example.internal/api/v1/push
    headers:
      X-Scope-OrgID: team-payments        # Mimir tenant
    queue_config:
      capacity: 20000
      max_shards: 50
      min_shards: 4
      max_samples_per_send: 5000
      batch_send_deadline: 5s
    write_relabel_configs:
      - source_labels: [__name__]
        regex: "go_gc_.*|promhttp_metric_handler_.*"
        action: drop                       # shed noise before it costs money

  - url: http://vminsert.example.internal:8480/insert/0/prometheus/api/v1/write
    queue_config:
      max_shards: 30

Two details in that file are easy to miss. The VictoriaMetrics URL embeds the tenant (0 is the account ID) in the path, which is how the cluster version does multi-tenancy; the single-node URL is just /api/v1/write. And the write_relabel_configs block is the cheapest cardinality control you own, because a sample dropped at the sender never reaches any backend or any bill.

The Thanos write path

In sidecar mode there is no hot write path to Thanos at all. Prometheus writes locally; roughly every two hours it cuts a block, and the sidecar uploads it. Until then the data is queryable only through the sidecar’s Store API. That is the weakness of sidecar mode: if the Prometheus pod and its disk vanish before the block is uploaded, you lose up to about two hours of data. Receive mode closes that gap with replicated ingestion, at the cost of running stateful receive pods with PVCs and a hashring configuration that must be kept consistent.

# Thanos Receive with a Ketama hashring, RF=3, writing blocks to S3
thanos receive \
  --grpc-address=0.0.0.0:10901 \
  --http-address=0.0.0.0:10902 \
  --remote-write.address=0.0.0.0:19291 \
  --tsdb.path=/var/thanos/receive \
  --tsdb.retention=6h \
  --label='receive_replica="$(POD_NAME)"' \
  --receive.hashrings-file=/etc/thanos/hashrings.json \
  --receive.hashrings-algorithm=ketama \
  --receive.replication-factor=3 \
  --receive.local-endpoint=$(POD_NAME).thanos-receive:10901 \
  --objstore.config-file=/etc/thanos/objstore.yml

The Mimir write path

Mimir’s distributor is where tenancy and limits are enforced. Each request carries a tenant ID in the X-Scope-OrgID header, and the distributor checks ingestion rate, burst size, label limits and active-series limits before forwarding to ingesters. If you run two Prometheus replicas for HA, Mimir can elect one replica per cluster via an HA tracker keyed on cluster and __replica__ labels, so only one stream is accepted. That is stored in a key-value store (memberlist by default, according to the configuration reference).

The classic architecture ties writes to ingester health: a write needs a quorum of ingesters, so rolling an ingester or losing a zone consumes your error budget. This is exactly what ingest storage addresses. With Kafka in the middle, the documentation says a write succeeds once Kafka commits it, ingesters leave the write path, and queriers need only one healthy ingester per partition. The price is a new stateful dependency (a Kafka-compatible cluster) that you must size, secure and monitor.

The VictoriaMetrics write path

vminsert is stateless apart from its in-memory buffers: it parses the incoming protocol, hashes each series and forwards it to vmstorage over a binary protocol. It accepts Prometheus remote write plus a long list of other inputs, among them OpenTelemetry metrics, InfluxDB line protocol, Datadog and Graphite. If a vmstorage node is down, vminsert reroutes its share to the remaining nodes so ingestion continues, and it rebalances later. That rerouting is why the project advises running more, smaller storage nodes: losing one of ten reshuffles about ten percent of the load instead of a third of it.

Why the difference matters

Think of the three as answers to “where does a sample first become durable?” In Thanos sidecar mode, the answer is Prometheus’ own disk. In Mimir classic it is a quorum of ingester WALs, and in Mimir ingest storage it is Kafka. In VictoriaMetrics it is the vmstorage disk, possibly replicated N ways. Each answer fixes what you must back up, what a node loss costs, and which component’s saturation shows up as remote write lag.

Query Path, Downsampling and Long-Range Reads

Ingestion gets the attention, but query behaviour is what users feel. A dashboard that spans 90 days at 15 second resolution would ask for around 518,000 samples per series, and a panel with a few hundred series becomes a hundred million sample reads unless something intervenes.

Figure 3: Query paths. Thanos Query fans out to sidecars and store gateways. Mimir splits queries in the query-frontend and reads ingesters plus store-gateways. VictoriaMetrics vmselect fans out to every vmstorage node.

Thanos: fan-out plus downsampling

The Thanos querier discovers Store API endpoints, sends each the time range and matchers, merges results and deduplicates series that differ only in replica labels. Store gateways keep an index header per block, so a query first narrows the candidate blocks by time range and external labels, then reads only needed chunks from the bucket, optionally through index and chunk caches such as Memcached.

Thanos is the only one of the three whose open-source edition documents built-in downsampling. The compactor creates 5-minute and 1-hour resolutions: raw blocks older than 40 hours are downsampled to 5 minutes, and 5-minute blocks older than 10 days are downsampled to 1 hour. The retention flags --retention.resolution-raw, --retention.resolution-5m and --retention.resolution-1h default to 0d, meaning forever. A crucial detail from the compactor docs: downsampling does not save space. It adds two additional blocks per raw block, so keeping all resolutions can grow storage roughly threefold. Its value is query speed over long ranges, and the Thanos querier can pick the resolution automatically or on request (Thanos compactor documentation).

Also note the singleton constraint: only one compactor may operate on a given stream of blocks in a bucket. Running two against the same stream causes overlapping blocks that need manual repair. In practice the compactor is also the component most often under-provisioned, because it builds output blocks on local disk and a large tenant needs hundreds of gigabytes of scratch space.

Mimir: sharded queries and split compaction

Mimir’s query-frontend splits a long range query into day-aligned sub-queries, executes them in parallel through the query-scheduler, and caches results. It can also shard a single query across series. This is what makes a 90-day dashboard bearable without any lower-resolution data. The trade-off is that every query reads raw samples: the compactor documentation does not mention downsampling at all, so Mimir relies on query-time parallelism and caching rather than pre-aggregated resolutions.

The compactor’s contribution is block hygiene. Compaction runs in a split-and-merge pattern: the split stage divides a tenant’s source blocks into groups, and the merge stage combines each shard down to a configurable number of output blocks. The documentation suggests about one shard per 8 million active series and warns that going beyond roughly 96 shards adds blocks, index headers and query fan-out. The default configuration disables the split stage, so large tenants must opt in. Retention is also disabled by default, so nothing is deleted until you set a period.

VictoriaMetrics: speed from the storage engine

vmselect asks every vmstorage node for matching data and merges the results. Because storage is columnar-ish and compressed, scanning is fast, and the project claims large storage and speed advantages over Prometheus-compatible peers in its own benchmarks. Those are vendor numbers measured on vendor-chosen workloads, so run your own comparison before believing a multiplier. The retention flag is -retentionPeriod (default one month, minimum 24 hours), and the docs say nodes should share the same value.

For long ranges, VictoriaMetrics offers downsampling and multi-level retention as features documented alongside its enterprise offering; confirm in the current docs which edition you need before designing around it. For open-source users the practical alternative is a recording-rule or vmalert pipeline that writes pre-aggregated series, which is also the pragmatic answer in Mimir.

Query language compatibility

Mimir and Thanos run Prometheus’ PromQL engine or a compatible one; Mimir’s own engine is described as fully PromQL compatible. MetricsQL, the VictoriaMetrics language, is PromQL-like with added functions and a few intentional semantic differences, such as handling of the lookbehind window and keeping metric names after some functions. Most dashboards run unmodified, but audit recording rules and alerts during migration, because subtle differences in rate window handling and staleness can change alert timing at the margin.

Multi-Tenancy, High Availability and Metrics Cardinality

Tenancy models

Mimir treats multi-tenancy as a first-class concern. Every request carries a tenant ID, and limits, compaction, caching and query scheduling are all per tenant. You can give each team its own ingestion rate, series cap and retention, and one team’s accidental label explosion trips that team’s limit instead of the cluster’s. Tenant isolation can also be strengthened with shuffle sharding, which assigns each tenant a subset of ingesters or queriers so a noisy tenant cannot touch every instance.

Thanos Receive supports tenants too. Tenants are created dynamically, each with its own TSDB, and the identity comes from a header (THANOS-TENANT by default, with default-tenant as the fallback) or from a TLS client certificate field. Tenancy there is a routing and storage separation, and it is less rich than Mimir’s per-tenant limit system. Sidecar mode has no tenancy at all beyond external labels.

VictoriaMetrics cluster identifies tenants as an accountID or accountID:projectID pair of 32-bit integers, taken from the URL path, from headers, or from per-sample labels. Tenants are created on first write and their data is spread across storage nodes. Be aware of what you are not getting: tenant isolation is data separation rather than a full quota system in the open-source version, so protect shared resources with vmagent-side limits and an authenticating proxy such as vmauth.

High availability, by component

Figure 4: Failure domains. Each system trades an extra stateful dependency for a different guarantee: object storage and compactor for Thanos, ingesters plus ring and optional Kafka for Mimir, replicated vmstorage nodes for VictoriaMetrics.

The cleanest way to compare HA is to list what holds state and what happens when it dies.

Thanos. State lives in Prometheus disks (recent), the object store (history) and, in receive mode, receive PVCs. The object store gives you very cheap, very durable history. The weak points are the singleton compactor (a stuck compactor stops retention and compaction but does not stop queries or ingestion) and the sidecar window between block uploads. Prometheus HA pairs are deduplicated at query time using a replica label, so you pay for both replicas’ storage unless you compact with vertical dedup, an advanced option the documentation flags as risky and irreversible.

Mimir. State lives in ingesters (recent), the object store (history) and the hash ring’s key-value store. Classic mode tolerates loss of a zone when zone-aware replication is on, because RF=3 across three zones leaves a quorum. It does not tolerate loss of two zones, and a bad rollout that restarts too many ingesters at once will cause write errors and read errors simultaneously. Ingest storage separates the failure domains, which is a real improvement but adds Kafka to the on-call rotation.

VictoriaMetrics. State lives on vmstorage local disks only. With replicationFactor=2 you survive one node loss and you need at least three nodes to keep full redundancy under that failure. There is no object store to be down, which is operationally pleasant, but you pay for replicated block or SSD storage, and capacity planning is on you. Backups go to object storage via vmbackup, which is a separate process you must schedule and, more importantly, test restoring.

Deduplication of HA scrape pairs

Running two Prometheus replicas that scrape the same targets is the standard way to avoid blind spots during restarts. The three systems handle the duplicates differently. Thanos Query deduplicates on read using the configured replica label. Mimir’s distributor can pick a leader replica and drop the other at write time via the HA tracker, which halves storage and ingestion cost. VictoriaMetrics deduplicates at query and merge time when you set -dedup.minScrapeInterval to your scrape interval; with -replicationFactor enabled the docs require -dedup.minScrapeInterval=1ms on vmselect to remove replica duplicates. Pick one deduplication strategy and document it, because stacking them is a common source of confusing gaps.

Cardinality: the actual limit

Most large Prometheus bills are cardinality bills. A request counter with labels for method (5 values), status (10), route (200) and pod (100 per deployment, churned daily) multiplies to 1,000,000 potential series per metric, and a histogram with 12 buckets multiplies that again by 14 when you include sum and count. A single carelessly labelled histogram can exceed the series budget of an entire team.

Where each system draws the line differs. Thanos has no central cardinality enforcement in sidecar mode; the limit is each Prometheus server’s memory, and Receive supports active-series limiting when run in router modes. Mimir enforces per-tenant limits in the distributor and ingesters (active series, ingestion rate, label counts and lengths), rejecting offending samples with explicit errors; the exact defaults vary by version, so read your version’s reference instead of trusting a blog. VictoriaMetrics documents a series limiter that can cap new series per hour and per day, and its cardinality explorer in the UI shows the biggest metrics and labels, which is where most investigations start.

The better answer is upstream. Use write_relabel_configs or vmagent relabeling to drop high-churn labels, set sample_limit and label_limit on scrape jobs so a bad exporter fails its own target rather than the backend, and review the top ten metrics by series count weekly. For a Kubernetes-heavy estate, cardinality control and workload sizing are connected; our guide to Kubernetes cost optimization and GPU rightsizing covers how observability volume feeds the cluster bill.

# Mimir per-tenant limits (illustrative values, verify field names for your version)
overrides:
  team-payments:
    max_global_series_per_user: 3000000
    ingestion_rate: 200000
    ingestion_burst_size: 400000
    max_label_names_per_series: 40
    compactor_blocks_retention_period: 395d

# VictoriaMetrics cluster: replicated, deduplicated, 13 months retention
vminsert -storageNode=vmstorage-0:8400,vmstorage-1:8400,vmstorage-2:8400 \
         -replicationFactor=2
vmselect -storageNode=vmstorage-0:8401,vmstorage-1:8401,vmstorage-2:8401 \
         -replicationFactor=2 -dedup.minScrapeInterval=1ms
vmstorage -retentionPeriod=13 -storageDataPath=/vm-data   # value in months

Cost and Capacity at 10M and 100M Active Series

Cost modelling for a time-series backend starts from arithmetic you can do on paper. All figures in this section are illustrative estimates from stated assumptions, not benchmarks. Replace the assumptions with your own measurements.

Sample rate and raw volume

At a 15 second scrape interval, each active series produces 4 samples per minute, or 5,760 per day. Ten million active series therefore ingest about 667,000 samples per second and 57.6 billion samples per day. One hundred million series ingest about 6.7 million samples per second and 576 billion per day.

Using the Prometheus documentation’s rule of thumb of one to two bytes per sample for its own TSDB, and assuming 1.5 bytes after block compaction, 10M series produce roughly 86 GB per day, or about 34 TB for 395 days (13 months). At 100M series that is about 340 TB. VictoriaMetrics claims materially better compression on its own benchmarks, so a conservative planning assumption is to test with your real data and expect somewhere between parity with Prometheus and a modest gain.

What grows beyond raw samples

Thanos with all three resolutions kept adds up to roughly three times raw storage according to its docs, so the 34 TB case could reach roughly 100 TB, although the extra blocks sit in cheap object storage. Replication multiplies hot-tier storage: Mimir RF=3 keeps three copies of the last few hours on ingester disks before upload, and VictoriaMetrics with replicationFactor=2 stores two full copies for the entire retention window on block storage. That last point is the hidden cost difference. Object-storage-backed designs pay replicated prices only for the head and unreplicated (but internally durable) prices for history, while local-disk designs pay replication on everything.

Memory is the other major driver. A reasonable planning range for in-memory head cost is several kilobytes per active series per replica; that is an assumption to validate, since it moves with label size and churn. At an assumed 8 KB per series, 10M series is about 80 GB per copy and about 240 GB across three Mimir ingester replicas, before headroom. At 100M series the same assumption gives 800 GB per copy and 2.4 TB across replicas, which explains why Mimir deployments at that scale need dozens of ingesters and why dropping a single useless label is worth real money.

Cost shape by system

Cost driver	Thanos	Mimir	VictoriaMetrics
Hot storage	Existing Prometheus disks	Ingester disks, 3 copies of recent data	Local disks, N copies of all data
Cold storage	Object store, up to about 3x raw with all resolutions	Object store, 1x plus compaction overhead	Optional backups to object store
Memory at scale	Prometheus servers plus store gateway caches	Ingesters dominate, then store-gateways	vmstorage uses OS page cache and modest RAM
Fixed overhead	Low to moderate	High (many components, caches, optionally Kafka)	Low
Scaling unit	Add Prometheus shards or receive pods	Scale each role independently	Add vmstorage nodes

Read that table as direction, not a quote. In many published migration stories the operational headcount saved by simpler systems outweighs the infrastructure delta, but those are anecdotes. A fair evaluation runs a dual-write pilot for two weeks with your real series and measures CPU, memory, disk and query latency per system.

Operational Burden, Migration and Remote-Write Compatibility

Operational burden is the dimension that rarely appears in benchmarks and dominates the first year. Thanos has the smallest conceptual step because it grows around Prometheus, but the component count creeps up: sidecars, querier, query-frontend, store gateways, compactor, ruler and optionally receive. Mimir has the most moving parts and the most mature tooling for running them (a Helm chart, mixins with dashboards and alerts, a runbook for each alert), and Grafana has invested heavily in the operator experience. VictoriaMetrics has the fewest parts in single-node mode, one process, and a modest three-role cluster, so the day-two story is often the easiest.

Migration paths

Because all three accept Prometheus remote write, the safest migration is dual writing. Add a second remote_write block, let both backends fill for as long as your longest alert or dashboard range needs, then switch Grafana’s datasource. For history you cannot wait for, each ecosystem offers backfill tooling: Thanos reads blocks directly from object storage, so you can upload existing Prometheus blocks with the bucket tools; Mimir ships mimirtool and a block backfill path for uploading TSDB blocks; VictoriaMetrics provides vmctl to migrate from Prometheus snapshots, Thanos, Mimir and others. Check each tool’s current flags in its documentation, since these utilities change between releases.

A migration order that rarely goes wrong: first run dual write and compare series counts and sample counts per metric between old and new; second, migrate alert rules and recording rules into the new ruler and diff firing history; third, move dashboards; fourth, cut over alerts; and only then remove local retention from Prometheus. Keep rollback open for two retention cycles of your slowest alert.

Remote-write compatibility notes

Remote write 1.0 is universally supported. Prometheus has also been stabilising remote write 2.0, which adds metadata, exemplars and native histograms in a more compact encoding. Support differs between receivers and moves quickly, so verify the specific version of the receiver you plan to run against the Prometheus release you use. Native histograms in particular have uneven backend support, and relying on them before your backend declares support will silently drop data. A cheap safety check is to compare prometheus_remote_storage_samples_failed_total and prometheus_remote_storage_samples_dropped_total to zero for a week before cut-over.

Finally, think about where recording rules run. Evaluating rules in the long-term system (Thanos Ruler, Mimir ruler, vmalert) reduces load on Prometheus and gives rules a global view, but a network partition between ruler and store can delay or fail evaluations, so alert on rule evaluation failures as well as on the alerts themselves.

Decision Matrix: Thanos vs Mimir vs VictoriaMetrics

The matrix below condenses the architecture differences into choices. The ratings are the author’s judgement from the documented designs, not measured scores.

Criterion	Thanos	Mimir	VictoriaMetrics
Fits existing Prometheus fleet unchanged	Best (sidecar)	Good (remote write)	Good (remote write)
Write path durability	Sidecar: block upload window. Receive: quorum	Quorum of ingesters, or Kafka with ingest storage	Replicated vmstorage nodes
Native multi-tenancy and per-tenant limits	Basic (Receive tenants)	Strongest	Tenant IDs, limited quotas
Built-in downsampling (open source)	Yes, 5m and 1h	No	Documented as enterprise feature
Object storage as primary history	Yes	Yes	No (local disk, backups to object store)
Component count	Moderate to high	High	Low
Query speed on long ranges	Good with downsampling	Good with sharding and caching	Very good in vendor tests (verify yourself)
Query language	PromQL	PromQL	MetricsQL (PromQL-like superset)
Best-fit team	Platform team with many Prometheus servers	Central observability team serving many tenants	Small team wanting low ops and high density

Choose Thanos when you have many existing Prometheus instances, want a global query view quickly, and your retention need is long but your ingestion centralisation need is low. Its sidecar mode is also the right pick when data residency means each cluster must keep its own raw data while allowing a global view.

Choose Mimir when you are building an internal metrics platform for many teams, need per-tenant isolation and limits, and have the platform capacity to run a microservices system. If you are already in the Grafana ecosystem, the integration with Grafana dashboards, Loki and Tempo is a genuine advantage.

Choose VictoriaMetrics when operational simplicity and density are the priority. Start single-node, graduate to the cluster only when ingestion or availability needs demand it, and decide early whether you need the enterprise features.

None of the three is a bad answer, which is why the argument in the community is so heated. The wrong answer is picking by logo and ignoring the shape of your problem.

Trade-offs, Gotchas, and What Goes Wrong

Cardinality bombs. The most common failure is not the database, it is a label. A deploy adds a user_id or request_id label to a counter and series count jumps by orders of magnitude within minutes. Mimir rejects samples over the tenant limit, which protects the cluster but silently loses data for that tenant. VictoriaMetrics’ limiter behaves similarly. Thanos sidecars simply run Prometheus out of memory. Alert on active series growth rate, not just absolute count.

Remote write backpressure. When a backend slows down, Prometheus’ remote write queue grows, memory rises, and eventually samples drop. The classic mistake is increasing max_shards without checking that the receiver can absorb the load, which converts a slow backend into a dead one. Scale the receiver first, then the sender.

Compactor starvation. For Thanos and Mimir, a compactor that cannot keep up leaves thousands of small blocks, which makes queries slower and costs more in object-store requests. Monitor compaction lag and block counts, and give the compactor real disk. For Thanos remember the singleton rule; for Mimir remember that the split stage is off by default.

Time and ordering. Blocks are immutable and time-indexed. Late or out-of-order samples need explicit configuration (Mimir has an out-of-order window setting and a write-behind log for it), and clock skew between Prometheus servers can create overlapping blocks that vertical compaction must clean up.

Object-storage surprises. Request charges, throttling on prefixes and eventual-consistency edge cases hit store gateways first. Use caches, keep block counts low, and test restore and bucket-failure scenarios.

Vendor-metric optimism. Every project publishes benchmarks where it wins. They are not dishonest, but they use workloads chosen by the author. Before committing, replay a day of your production traffic against two candidates and compare resource use and p95 query latency on your own dashboards.

Lock-in is mild but real. Remote write is portable. Rule syntax, tenant identifiers, dashboards that use MetricsQL-only functions, and retention semantics are not. Keep your dashboards PromQL-clean if you want an exit.

Practical Recommendations

Start by measuring. Count active series with prometheus_tsdb_head_series, ingestion with rate(prometheus_tsdb_head_samples_appended_total[5m]), and list the top metrics by series count. If the total is under a few million series and a single replica pair covers your availability needs, a single-node VictoriaMetrics instance or a handful of Prometheus servers with Thanos sidecars is usually enough, and either is a day’s work.

If you serve many teams and need isolation, accept Mimir’s complexity and run it from the official Helm chart with its monitoring mixin from day one. If you value the smallest operational surface, choose VictoriaMetrics, but verify which features you need are in the edition you will run. If you already operate many Prometheus servers across clusters and regions, sidecars with a global querier are the quickest win.

Whichever you pick, this checklist applies:

Dual-write for at least one full retention cycle of your slowest alert before cutting over.
Drop unused metrics and high-churn labels at the sender with write relabeling.
Set sample_limit and label_limit on every scrape job.
Decide one HA deduplication mechanism and document it.
Alert on remote write lag, dropped samples, compaction lag and active series growth.
Keep recording rules for expensive long-range dashboards, whatever the engine.
Test a full restore from backup or bucket, not just a successful upload.
Re-run the cost model each quarter with measured, not assumed, bytes per sample and bytes per series.

When the platform is observable, the cluster costs are visible too. If your fleet includes edge or industrial sites, the collection layer matters as much as the store, and our comparison of the OpenTelemetry Collector, Vector and Fluent Bit shows where to filter and route before samples ever reach your long-term store.

Frequently Asked Questions

What is the best Prometheus long term storage in 2026?

There is no single best. Thanos suits teams with many existing Prometheus servers who want a global view with object storage. Mimir suits multi-tenant platforms that need strict per-tenant limits. VictoriaMetrics suits teams that value low operational overhead and storage density. Choose by tenancy needs, series count, team size and whether you can run object storage and stateful ingesters, then validate with a dual-write pilot on your own traffic.

Is Thanos or Mimir better?

Thanos is lighter to adopt because its sidecars leave your Prometheus servers in place and it adds a global query layer. Mimir is heavier but delivers a fully managed-style ingestion service with per-tenant limits, query sharding and, from version 3.0, an optional Kafka-based ingest storage design. Pick Thanos for federation of existing servers, Mimir for a central multi-tenant platform.

Is VictoriaMetrics a drop-in replacement for Prometheus storage?

For most dashboards, yes: it accepts Prometheus remote write and speaks PromQL-compatible queries through MetricsQL. It is not literally identical, since MetricsQL changes some semantics and adds functions, so audit alert rules and recording rules during migration. It stores data in its own format rather than Prometheus TSDB blocks, so migration of history uses a tool such as vmctl rather than copying files.

How much does Prometheus long-term storage cost?

It depends on active series, scrape interval, retention and replication. With 1 to 2 bytes per sample as a planning figure, 10 million series at a 15 second interval produce about 86 GB per day at 1.5 bytes per sample, roughly 34 TB over 13 months. Memory for the in-memory head usually costs more than cold storage, so cardinality control saves the most money.

Do I need downsampling?

Only if you query very long ranges frequently and raw reads are slow. Thanos provides 5 minute and 1 hour downsampling, but its documentation notes that it adds storage rather than saving it, since the downsampled blocks exist alongside raw ones. Mimir relies on query splitting, sharding and caching instead, and recording rules cover most dashboard needs on any system.

Can I run Prometheus remote write to two backends at once?

Yes. Prometheus supports multiple remote_write entries, each with its own queue, so you can dual-write to compare backends or migrate safely. Each endpoint adds memory and CPU overhead on the sender, so monitor the queue metrics. Add relabeling per endpoint if one backend should receive only a subset of series.