OTLP vs Prometheus Remote Write: The 2026 Metrics Pipeline Decision

Every platform team that has run Prometheus past a few hundred nodes eventually hits the same wall: a label combination nobody reviewed in code review quietly multiplies the number of active series by 40x, remote_write queues start backing up, and someone gets paged because the metrics backend — not the service it’s monitoring — is the thing that’s down. Now that OpenTelemetry’s metrics signal is stable and shipping in every major SDK, the question platform teams actually have to answer in 2026 isn’t “should we adopt OpenTelemetry” — that ship sailed for traces and is sailing fast for logs — it’s narrower and more consequential: for metrics specifically, do you standardize your transport on OTLP, keep Prometheus remote write, or run both through a bridge indefinitely? This is the core of the otlp vs prometheus remote write decision, and it is genuinely an ADR-worthy call, not a preference. What this covers: the architectural difference between OTLP’s push model and Prometheus’s pull-then-remote-write model, the cardinality risk profile of each, the delta-vs-cumulative temporality trap that burns teams migrating their first OTLP metrics pipeline, the OTel Collector’s Prometheus receiver/exporter bridge as a practical migration path, and where Mimir, Cortex, and Thanos each stand on native OTLP ingestion today.

Context and Background

Prometheus’s pull model has been the default metrics architecture in Kubernetes-native shops since roughly 2016: an agent scrapes /metrics endpoints on a timer, the Prometheus server holds a local TSDB, and remote_write ships samples to a long-term store like Mimir, Cortex, or Thanos for durability and cross-cluster querying. It’s a design with real virtues — service discovery makes target management nearly free, a dead target is trivially detectable as a scrape failure, and the protocol has had a decade to get boring in the good sense.

OpenTelemetry approached the same problem from a different constraint: it needed one wire protocol and one SDK surface that could carry traces, metrics, and logs with consistent semantic conventions, correlated by shared resource attributes and trace context. OTLP metrics are pushed from the SDK (or an intermediate Collector) directly to a backend, encoded as protobuf over gRPC or HTTP, following the same OpenTelemetry metrics data model used by every language SDK. As of the OpenTelemetry Specification 1.60.0 release, metrics are marked stable across the core spec and in the overwhelming majority of language implementations — this is no longer an early-adopter bet. See the OpenTelemetry specification status page for the current per-signal, per-language breakdown before you commit a fleet.

The reason this now reads as a genuine fork in the road, rather than “just adopt OTel everywhere,” is that Prometheus itself has moved. Prometheus 3.0 (November 2024) added a native OTLP ingestion endpoint behind --web.enable-otlp-receiver, and the Prometheus team shipped Remote Write 2.0 as an experimental spec with native histogram support, metadata, and exemplars carried in-band. Both ecosystems are now converging on each other’s territory, which is exactly what makes the decision non-obvious: you’re no longer choosing “old vs new,” you’re choosing which protocol is the source of truth for your fleet’s metrics contract, with the other treated as a compatibility surface. For platform teams already running the Grafana Alloy / OpenTelemetry Collector stack for traces and logs, this decision also determines whether metrics get to ride the same pipeline or need a parallel one.

The urgency behind this decision isn’t abstract. Platform teams that delayed picking a canonical transport for two or three years now have both patterns embedded across dozens of services, each with its own cardinality profile, its own dashboards, and its own on-call runbooks written against whichever protocol happened to be default when that service was scaffolded. Every additional quarter without a documented decision adds services to both camps and makes the eventual consolidation more expensive — this is the same compounding-debt dynamic teams have seen with logging formats, service meshes, and CI systems, and metrics pipelines are not exempt from it. Writing this decision down as an ADR, even an informal one, is cheap insurance against re-litigating the same argument every time a new team lead joins.

Deciding the Metrics Transport Standard: OTLP or Remote Write

Direct answer: for greenfield services instrumented with OpenTelemetry SDKs, emit OTLP metrics natively and route them through a Collector that fans out to your backend — this keeps one instrumentation surface for all three signals and gives you cardinality and temporality control in one place. For an existing Prometheus fleet with thousands of /metrics exporters you don’t control (node_exporter, kube-state-metrics, database exporters), keep scraping via Prometheus and bridge into OTLP only at the Collector boundary, rather than trying to re-instrument every third-party exporter.

Decision: transport standard for the metrics pipeline

Context. A platform team running 400+ microservices across three Kubernetes clusters needs one metrics ingestion contract for the next three years. Half the services are already OpenTelemetry-instrumented for tracing; the other half are legacy Java and Go services exposing Prometheus /metrics. Leadership wants a single pane of glass and a predictable cost curve as headcount and service count both grow roughly 30% year over year.

Options considered.

1. Prometheus pull + remote_write everywhere. Keep the incumbent model. Every service exposes a /metrics endpoint; Prometheus (or an Alloy/Grafana Agent fleet) scrapes on an interval and remote-writes to Mimir. OTLP traces and logs run on a separate pipeline.
2. OTLP push everywhere, Prometheus retired. Every service, including legacy ones, gets an OTel SDK or a prometheus_receiver-fed Collector that converts scrapes to OTLP internally, and all signals — traces, metrics, logs — travel the same OTLP pipeline to the same backend.
3. Hybrid: OTLP standard for new services, Collector bridge for the legacy scrape fleet. New and newly re-instrumented services emit OTLP metrics directly. The Collector’s prometheusreceiver continues scraping legacy exporters and normalizes them into the OTel metrics data model before export, so the backend only ever ingests one shape of data.

Decision. Option 3. Standardize the backend contract on OTLP’s data model, but don’t force a rip-and-replace of every scrape target. The Collector becomes the seam: it speaks Prometheus’s pull dialect outward, toward legacy exporters, and OTLP inward, toward the backend and the rest of the observability pipeline.

Consequences. You get one semantic model (resource attributes, consistent metric naming, shared trace-metric-log correlation) without a multi-quarter re-instrumentation project. The cost is operational: the Collector’s prometheusreceiver → OTLP conversion path has real edge cases (see the temporality section below), and you now operate a Collector fleet as a stateful-ish component in the critical path of every metric, which it wasn’t before.

Figure 1: Push (OTLP) vs pull (Prometheus scrape + remote_write) — the two architectures converge at the long-term storage layer but differ fundamentally in where backpressure and service discovery live.

The push/pull distinction in Figure 1 is not cosmetic — it changes where failure shows up. In the pull model, a target that’s down or overwhelmed simply fails a scrape; Prometheus’s up metric tells you immediately, and the failure is isolated to that target. In OTLP’s push model, backpressure lives at the exporter and the Collector’s receiver queue: if the backend is slow, the SDK’s OTLP exporter buffers, retries, and eventually drops — and unlike a missed scrape, a dropped push is silent unless you’re specifically monitoring the Collector’s own otelcol_exporter_send_failed_metric_points counter. Teams moving from pull to push consistently underestimate how much free debuggability they got from “a dead target is a failed scrape” and have to build equivalent visibility into Collector health explicitly.

Cardinality: the axis that decides real-world cost

Cardinality — the number of unique time series produced by a metric name plus its label/attribute value combinations — is the single biggest cost and reliability driver in any metrics pipeline, and the two protocols default to opposite failure modes.

Prometheus’s cardinality risk is well understood because the ecosystem has ten years of scar tissue: a label like user_id or a raw URL path attached to a counter will blow up active series count, and remote_write queues will back up under the write-amplification. The mitigation playbook is mature — metric_relabel_configs to drop or aggregate high-cardinality labels before scrape, recording rules to pre-aggregate, and per-tenant series limits enforced at the Mimir/Cortex ingester.

OTLP’s cardinality risk is structurally similar but less battle-tested operationally, because OpenTelemetry’s semantic conventions actively encourage rich resource attributes — k8s.pod.name, k8s.deployment.name, service.instance.id, cloud.availability_zone — attached to every metric point by default via the Collector’s resourcedetection processor. That’s a feature for correlating metrics with traces and logs, and a cardinality landmine if you don’t prune it before export. The practical control point is the Collector’s attributes and filter processors, which need to run before the OTLP exporter, not after — there’s no equivalent of Prometheus’s server-side series limit as a last line of defense once the data has left the Collector and hit a backend that isn’t cardinality-aware.

Temporality: the gotcha nobody’s postmortem warns you about until it happens

This is the trap. OpenTelemetry metrics carry an explicit aggregation temporality — delta or cumulative — and it is set per-instrument by the SDK, not negotiated with the backend. Cumulative temporality reports a running total since the process started (this is what Prometheus’s rate() function expects). Delta temporality reports the change since the last export interval (this is what backends like Datadog prefer, and it’s the default some OTel SDKs ship with for language ergonomics reasons).

The gotcha: if a Collector or backend receiving OTLP metrics expects cumulative sums and gets delta, the Prometheus-compatible exporter in the Collector does not error loudly — it either converts the sum to a gauge (silently losing rate()-ability) or drops delta histograms outright, depending on version and configuration. A service that looks perfectly healthy in its OTLP-native dashboard can show completely wrong or empty data once bridged into a Prometheus-shaped backend, and the failure is a data-quality bug, not an outage — it doesn’t page anyone. The fix is explicit: set OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=cumulative at the SDK level for any service whose metrics need to reach a Prometheus-model backend, and verify it in a staging environment before rollout, because the default varies by SDK and language.

Operational maturity: what you give up and what you gain

The honest accounting here has to include tooling maturity, not just architecture. PromQL has ten years of accumulated recording rules, alerting patterns, capacity-planning dashboards, and institutional muscle memory across nearly every platform team that has run Kubernetes at scale. OTLP’s query-side story is inherited entirely from whatever backend you pick — there is no equivalent of PromQL that ships with the protocol itself, which means the actual query experience for OTLP-sourced metrics depends on whether your backend translates the OTel data model back into something PromQL-compatible (as Prometheus 3.0 and Mimir do) or exposes a separate query language. Teams underestimate this gap because instrumentation and querying feel like adjacent concerns, but they’re the two halves of the same operational cost, and only one of them (instrumentation) is what OTLP actually standardizes.

The gain, symmetrically, is real: once metrics, traces, and logs share the same resource schema, an on-call engineer can pivot from a spiking http.server.duration metric to the exact traces contributing to that spike without hand-matching labels across three separate systems. That correlation is the actual product OpenTelemetry is selling, and it’s most valuable exactly in the incident-response moments where a Prometheus-only stack forces manual cross-referencing between Grafana panels and a separate tracing UI.

Deeper Analysis: Where the Two Protocols Actually Diverge

Beyond the headline push-vs-pull framing, four concrete axes determine which protocol wins for a given team, and they don’t all point the same direction.

Figure 2: The Collector-as-bridge migration pattern — Prometheus receiver in, OTLP data model out, one Collector fleet serving both old and new instrumentation.

The Collector shown in Figure 2 is the practical migration path almost every real fleet takes, because it lets you flip the default for new services to OTLP without a forced migration of legacy exporters. The prometheusreceiver component scrapes existing /metrics endpoints exactly like a normal Prometheus server would — same service discovery, same relabeling — and converts each scrape into OTLP’s metrics data model internally, including a best-effort temporality assignment (Prometheus counters become cumulative sums, which maps cleanly). Downstream, a single otlp or prometheusremotewrite exporter fans out to your backend. This is also the pattern the OpenTelemetry project documents directly in its “Prometheus and OpenTelemetry — better together” guidance, and it’s the same seam used by teams unifying logs into the same OTel pipeline.

Figure 3: Cardinality-control decision flow — the earlier you drop a high-cardinality attribute, the cheaper it is; SDK-side is free, Collector-side costs CPU, ingester-side costs a rejected write and a confused on-call engineer.

Decision matrix: OTLP metrics vs Prometheus remote write

Axis	OTLP metrics	Prometheus remote write
Cardinality handling	No server-side limit by default; controlled via Collector attributes/filter processors pre-export. Rich resource attributes are the default, so unmanaged growth is the common failure.	Mature tooling: metric_relabel_configs, recording rules, and per-tenant series limits at the Mimir/Cortex ingester act as a hard backstop even if scrape config is sloppy.
Push vs pull	Push from SDK or Collector. No service discovery needed for the send path, but a dead exporter is silent unless the Collector’s own health metrics are monitored.	Pull via scrape. Target liveness is free (up{} metric); service discovery (Kubernetes SD, Consul, file SD) is a first-class, mature feature.
Protocol overhead	Protobuf over gRPC or HTTP/protobuf; carries full resource + attribute context per point, larger payloads but richer correlation with traces/logs.	Remote Write 1.0 is a flat protobuf WriteRequest; Remote Write 2.0 (experimental) adds metadata, exemplars, and native histograms in-band, closing much of the gap.
Ecosystem maturity	Native ingestion in Prometheus 3.0 (--web.enable-otlp-receiver) and Grafana Mimir (OTLP over HTTP). Cortex and Thanos support is bridged via the Collector’s prometheusremotewriteexporter, not always ingested natively.	A decade of tooling: PromQL, alerting rules, every major dashly/Grafana integration, and every long-term-storage backend (Mimir, Cortex, Thanos) treats it as the first-class native format.
Temporality model	Explicit delta/cumulative per instrument; must be configured correctly for the downstream consumer or data silently degrades (sums→gauges, delta histograms dropped).	Implicitly cumulative; PromQL’s rate()/increase() assume it, no equivalent trap exists natively.
Semantic correlation	Shared resource schema with traces and logs (service.name, k8s.pod.name) out of the box — this is OTLP’s core value proposition.	Requires manual label conventions matched by hand across metrics, logs, and traces if you want cross-signal correlation.

A representative Collector config bridging a legacy Prometheus target into an OTLP-shaped pipeline, with cardinality pruning and temporality forced to cumulative:

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'legacy-services'
          kubernetes_sd_configs:
            - role: pod

processors:
  attributes/drop_highcard:
    actions:
      - key: http.target
        action: delete
      - key: user_id
        action: delete
  batch:

exporters:
  otlp:
    endpoint: otel-backend:4317
    tls:
      insecure: false

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      processors: [attributes/drop_highcard, batch]
      exporters: [otlp]

And the SDK-side flag every migrating service needs set explicitly, rather than relying on the language default:

export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=cumulative

On ecosystem maturity specifically: Prometheus 3.0 shipped with a native OTLP receiver, meaning you can point an OTLP exporter directly at a Prometheus server without a Collector in between for simple cases. Grafana Mimir accepts OTLP over HTTP natively as of its current releases, per Grafana’s own OpenTelemetry ingestion documentation. Thanos and Cortex, by contrast, are largely still consumed through the Collector’s prometheusremotewriteexporter bridge rather than native OTLP ingestion — if your long-term store is one of those two, budget for the Collector as a permanent part of the architecture, not a transitional shim you’ll remove in a year.

On protocol overhead specifically, the difference is measurable rather than theoretical. A Prometheus remote_write WriteRequest carries each sample as a timestamp, a float64 value, and a flat set of string labels — compact by design, since the format predates any expectation of carrying cross-signal context. An OTLP metric point, by comparison, carries the same core sample plus a resource object (potentially a dozen attributes describing the host, container, cloud region, and Kubernetes topology) and an instrumentation-scope identifier, repeated per point unless the transport-level batching and string interning in the Collector deduplicate it efficiently. In practice this means an OTLP payload for an equivalent set of samples is commonly 1.5x to 3x larger on the wire than the same data serialized as a Prometheus WriteRequest, before compression — and after gRPC’s built-in compression, the gap narrows but doesn’t disappear. Remote Write 2.0 closes some of this gap on the Prometheus side by adding string interning of its own, which is worth checking before assuming the overhead comparison is fixed in OTLP’s favor for correlation value alone.

Trade-offs, Gotchas, and What Goes Wrong

The single most common failure mode reported by teams migrating is exactly the temporality trap above, compounded by the fact that it fails quietly. A second, subtler failure: Collector batch processor misconfiguration under load causes OTLP export queues to grow unbounded, and unlike a Prometheus scrape timeout (which is visible as a gap in the graph), a backed-up OTLP exporter queue just delays data arrival — dashboards look fine, on-call just doesn’t notice metrics are 20 minutes stale until an incident review.

Resource attribute cardinality is the third recurring gotcha. Teams that enable resourcedetection processors for Kubernetes, cloud provider, and host metadata without an explicit allowlist routinely see 5-10x active series growth in the first week after cutover, because every metric point now carries k8s.pod.name (which is high-cardinality by construction — pods churn) alongside the metric’s own labels. This is not a bug in OpenTelemetry; it’s the direct, foreseeable consequence of a design that optimizes for cross-signal correlation by default. The fix is proactive attribute pruning in the Collector pipeline, not a reactive cardinality-limit alarm after the ingester starts rejecting writes.

A fourth gotcha specific to hybrid deployments: mixing native Prometheus remote_write from some services with OTLP from others, into the same backend, without normalizing metric naming conventions between the two, produces duplicate-looking series with subtly different label sets (pod vs k8s.pod.name) that break dashboards and alerting rules built against one convention when the other convention’s data shows up. Pick one naming convention as canonical during the migration and translate at the Collector boundary — don’t let both conventions reach the query layer.

Finally, cost: OTLP’s richer per-point payload (full resource attributes on every metric point, not just the labels that matter to that metric) means naive migrations see a real increase in both network egress and backend storage cost compared to an equivalently-scoped Prometheus remote_write stream, until attribute pruning is tuned. Budget a cost-monitoring pass in week one of rollout, not week twelve.

A fifth, less obvious gotcha shows up specifically in histogram-heavy services. OpenTelemetry’s default histogram aggregation is explicit-bucket, matching Prometheus’s classic histogram shape, but OTel SDKs increasingly default new instruments to exponential (base-2) histograms for better resolution with fewer buckets. Not every backend and not every Collector exporter version handles exponential histograms identically when bridging to a Prometheus-shaped consumer — some flatten them into a fixed bucket approximation that changes your p99 latency numbers by a meaningful margin compared to what the exponential histogram actually recorded. If your SLOs are defined against tail-latency percentiles, verify the histogram type end to end through your actual pipeline before trusting a migrated dashboard’s p99, rather than assuming a “histogram is a histogram” equivalence.

Practical Recommendations

For a team making this call in the second half of 2026, the pragmatic path is rarely all-or-nothing. Standardize on the OTLP data model as your backend contract if you’re running (or building toward) a unified traces/metrics/logs pipeline — the correlation value is real and compounds as your service count grows. But don’t force legacy Prometheus exporters to be rewritten; bridge them through the Collector’s prometheusreceiver, which is mature and well-documented. Treat cardinality control as a pipeline-stage concern from day one in both models — it’s cheaper to drop a label at the SDK or Collector than to discover it at the ingester three weeks into a cost overrun.

Checklist before cutting over any service’s metrics pipeline:

• [ ] Confirm the target backend’s native ingestion path (Prometheus 3.0 and Mimir accept OTLP natively; Thanos/Cortex likely need the Collector bridge)
• [ ] Set OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=cumulative explicitly on every SDK feeding a Prometheus-model backend — do not rely on the language default
• [ ] Audit resourcedetection and other automatic attribute processors for cardinality impact before enabling in production
• [ ] Establish one canonical metric-naming and label convention across OTLP and remote_write sources before both reach the same backend
• [ ] Instrument the Collector fleet itself (otelcol_exporter_send_failed_metric_points, queue depth) — a push pipeline needs its own health signal, unlike pull’s free up{} metric
• [ ] Run a cost pass in week one comparing OTLP payload size against the equivalent Prometheus remote_write stream, and prune before it’s a budget conversation

If your organization is still early — a handful of services, one cluster, no cross-team dependency on existing dashboards — there’s a real case for skipping the hybrid phase entirely and starting OTLP-native, using Prometheus’s own native OTLP receiver or Mimir’s OTLP endpoint as the ingestion point. The hybrid Collector-bridge pattern earns its complexity specifically at the scale where a full re-instrumentation isn’t feasible in a single quarter.

Whichever path you pick, treat the decision as reversible-but-costly rather than permanent-and-final. Write it down as a short ADR — context, options considered, decision, consequences, exactly as this post is structured — and revisit it on a fixed cadence (annually is reasonable) rather than letting it drift by default as new services get scaffolded against whatever the last engineer happened to copy-paste. The actual switching cost, if you get the initial cardinality and temporality controls right, is dominated by re-pointing dashboards and alerting rules, not by re-instrumenting application code — which is exactly why getting the Collector-boundary contract right early matters more than getting the SDK-level choice right on day one.

Frequently Asked Questions

Is OTLP going to replace Prometheus remote write entirely?

Not in the near term, and probably not fully ever. Prometheus’s pull model and PromQL ecosystem are too embedded in existing tooling, dashboards, and alerting rules for most fleets to justify a full rip-and-replace. The more likely trajectory, visible in Prometheus 3.0’s native OTLP receiver and Remote Write 2.0’s added metadata support, is convergence: Prometheus absorbing OTLP as an ingestion option while remote_write picks up features (exemplars, native histograms) that narrow the gap with what OTLP already carries. Expect both protocols to remain first-class citizens in the Kubernetes observability ecosystem for years, with the practical decision being which one is canonical for your organization rather than which one wins outright.

Do I need the OpenTelemetry Collector if I’m only migrating metrics, not traces or logs?

Practically, yes, unless every service you own is already instrumented with an OTel SDK and your backend accepts OTLP natively. The Collector is what lets you bridge legacy Prometheus /metrics exporters into the OTLP data model without rewriting them, and it’s also where you enforce cardinality controls and temporality normalization centrally instead of per-service. Running it purely for metrics is a legitimate, common deployment pattern.

What is aggregation temporality and why does it break my dashboards after migration?

Temporality determines whether an OTLP metric reports a running cumulative total or the delta since the last export. Prometheus and PromQL’s rate() function assume cumulative counters. If your OTel SDK defaults to delta (common in some language implementations) and you don’t override it, the Collector’s Prometheus-compatible exporter will convert delta sums to gauges or drop delta histograms, and dashboards built for counters silently break or read empty — without triggering any alert, because nothing errored.

How does OTLP cardinality risk compare to Prometheus’s, in practice?

Structurally similar risk, different default exposure. Prometheus’s risk comes from careless label choices in application code or scrape configs, and it has mature server-side backstops (relabeling, per-tenant ingester limits). OTLP’s risk comes from resource attributes attached automatically by Collector processors like resourcedetection, which are genuinely useful for correlation but have no equivalent server-side backstop by default — the control point is earlier, in the Collector pipeline, and teams that don’t proactively prune there see faster, less visible series growth.

Which long-term storage backend should I pick if I’m standardizing on OTLP?

Grafana Mimir currently has the most mature native OTLP-over-HTTP ingestion among the open-source long-term stores, per Grafana’s own documentation. Prometheus 3.0 itself can act as a native OTLP receiver for smaller deployments that don’t need a separate long-term-storage tier. Thanos and Cortex are generally reached via the Collector’s prometheusremotewriteexporter bridge rather than native OTLP ingestion as of 2026, so factor that extra hop into latency and operational-complexity budgeting if either is your existing backend. If you’re already running Thanos or Cortex at scale and switching backends isn’t on the table this year, the Collector bridge is a fully supportable long-term architecture, not just a stopgap — plenty of large fleets run it as their permanent shape.

Can I run both protocols permanently instead of picking one as canonical?

You can, and many large organizations effectively do during multi-year migrations, but it’s worth being deliberate about it rather than letting it happen by default. The real risk of a permanent hybrid isn’t the Collector complexity — it’s dashboards and alerting rules built against one naming/label convention silently failing to match data that arrives via the other convention. If you go hybrid long-term, standardize label and metric-naming conventions at the Collector boundary so the query layer only ever sees one shape of data, regardless of which protocol produced it.

Further Reading

• Grafana Alloy and OpenTelemetry Collector tutorial for 2026
• Continuous profiling with eBPF in 2026
• OpenTelemetry logs and the unified telemetry pipeline in 2026
• OpenTelemetry Specification Status Summary
• Prometheus Remote-Write 2.0 specification (experimental)

Figure 4: Temporality handling sequence — the delta-to-cumulative conversion has to happen explicitly at the Collector or SDK boundary, or the backend silently receives gauges instead of counters.

By Riju — about