W3C Trace Context and Baggage: How Distributed Tracing Context Propagates Across Services and Queues (2026)

A distributed trace is only as good as its weakest hop. You can instrument every service perfectly, ship every span to a capable backend, and still open a trace view that shows a lonely gateway span with nothing beneath it, while the real work appears as three unrelated orphan traces somewhere else. Nothing is wrong with the spans. What failed is the thread that was supposed to tie them together: a few dozen bytes of identifiers that must survive every process boundary, thread switch, proxy and message broker between the user’s click and the last database write.

That thread is the subject of distributed tracing context propagation, and in 2026 it has a standard vocabulary: the W3C traceparent and tracestate headers for identity, the W3C baggage header for application data, and the OpenTelemetry Context and Propagator APIs for moving both inside and between processes. The standards are small. The failure modes are not.

What this covers: the exact wire format of traceparent, tracestate and baggage; how OpenTelemetry injects and extracts them; in-process pitfalls in async, thread-pool and goroutine code; propagation over HTTP, gRPC and message brokers, including when to use span links instead of parent-child; the trust-boundary and privacy problems of baggage; legacy B3 and Jaeger formats; sampling decisions at the edge; and a systematic way to debug a broken trace.

Context and Background

Before standard headers, every tracing system invented its own. Zipkin used the X-B3-* family, Jaeger used uber-trace-id, AWS X-Ray used X-Amzn-Trace-Id, and each commercial APM agent had a proprietary header. This worked while a trace stayed inside one vendor’s agents. It collapsed the moment a request crossed from a service instrumented by one tool into a managed gateway, a SaaS API or a cloud load balancer that spoke a different dialect: the foreign component either dropped the unfamiliar header or forwarded it blindly without joining the trace.

The W3C Distributed Tracing Working Group standardised the answer. The Trace Context specification defines traceparent and tracestate; the companion Baggage specification defines a separate header for user-defined key-value pairs. Both are published by the W3C, and both are the default propagation formats of OpenTelemetry. At the time of writing, the Trace Context Level 2 document I reviewed was a Candidate Recommendation Draft dated 28 March 2024, and the Baggage document a Candidate Recommendation Snapshot dated 30 May 2024; verify the current maturity level of each on w3.org before citing it in a compliance context, because status labels move.

OpenTelemetry (OTel) layers an API on top: a Context object that carries the active span and baggage inside a process, and Propagator objects that serialise that context into a carrier (an HTTP request, a gRPC metadata map, a Kafka record’s headers) and read it back out. The OTel specification is explicit that the API uses no-op propagators unless they are configured, which is a polite way of saying that if nobody wires up propagation, your traces silently fragment. SDKs normally configure sensible defaults, but the guarantee belongs to the SDK, not the API.

Two related posts on this site cover adjacent decisions. If your next question after “does my trace connect?” is “which of these connected traces do I keep?”, read tail-based sampling in the OpenTelemetry Collector, because the sampling decision rides in the very header discussed here. And because gRPC metadata is one of the carriers we cover, the transport trade-offs in gRPC vs REST vs GraphQL vs Connect are a useful companion.

The primary sources used throughout are the W3C Trace Context Level 2 and Baggage documents, the OpenTelemetry Propagators API and the OpenTelemetry messaging span conventions.

The Reference Model: Identity in Headers, Context in Process

Distributed tracing context propagation works in two layers. Across the network, a request carries a traceparent header (trace ID, parent span ID, flags), an optional tracestate header (vendor data) and an optional baggage header (application data). Inside each process, an OpenTelemetry Context object holds the current span and baggage, and propagators convert between that object and the headers on every inbound and outbound call.

Figure 1: Distributed tracing context propagation. Headers carry identity across hops, the in-process Context carries it between function calls, and a gateway at the trust boundary sanitises what comes in.

The diagram shows the lifecycle in one picture. A client request arrives at a gateway, which decides whether to trust any inbound trace headers. Service A extracts the context, starts a server span, and keeps the resulting context active while it works. Every outbound call, whether an HTTP request to Service B, a database query or a Kafka produce, reads the active context and injects a fresh traceparent whose parent ID is the span making the call. The consumer on the far side of Kafka extracts the header from the record and links its processing span back to the producer.

Three properties make this model robust. First, the headers are stateless: no service needs a registry or sidecar to understand them, so any component that forwards headers unchanged keeps the trace alive. Second, identity and data are separated: traceparent is small and fixed-shape so every hop can afford it, while baggage is open-ended and therefore bounded and treated with suspicion. Third, the in-process context is the single source of truth: instrumentation libraries do not pass trace IDs through function arguments, they read the ambient context, which is why losing the ambient context is the number one cause of broken traces.

The traceparent header, field by field

A version 00 header looks like this:

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

It has four dash-separated fields. The version is two lowercase hex characters, and ff is invalid. The trace ID is 32 lowercase hex characters (16 bytes), and all zeros is invalid. The parent ID, which the spec also calls the span ID of the caller, is 16 lowercase hex characters (8 bytes), also invalid when all zeros. The flags field is 8 bits written as two hex characters. A version 00 header is exactly 55 characters long, which is why a parser can reject malformed input with a cheap length check before it looks at any content.

Figure 2: Anatomy of the three propagation headers. traceparent is fixed-shape and mandatory; tracestate and baggage are optional lists with different limits and different owners.

The trace ID is generated once at the root of the trace and never changes. The parent ID changes at every hop: when Service A calls Service B, A puts the ID of its outgoing client span in the header, and B’s server span records it as its parent. That single rule is what lets a backend reassemble the tree from a bag of independently reported spans. There is no central coordinator; the tree is implied by parent pointers.

The flags byte: sampled and random

Bit 0 (00000001) is the sampled flag. The spec describes it as the caller’s recommendation, not an order: a downstream service may legitimately make its own sampling choice, though head-sampling designs usually honour it so that traces are complete or absent rather than half-present. Bit 1 (00000010) is the random trace ID flag introduced in Level 2. A participant that starts a new trace and sets it asserts that at least the rightmost 7 bytes of the trace ID are uniformly random, which lets downstream samplers hash a slice of the ID and get an unbiased decision. Services that continue a trace must preserve the flag as received. All other bits are reserved and must be zero.

The practical lesson is to test bits, not bytes. A value of 03 means sampled and random; code that compares the whole flags byte to 01 will treat a perfectly valid sampled trace as unsampled once Level 2 flags appear. This is a classic compatibility bug that only surfaces when a different vendor’s library joins the path.

tracestate: vendor data with hard limits

tracestate is a comma-separated list of key=value members that lets vendors attach their own data without inventing a new header, for example a vendor’s own sampling priority or an account identifier. The Level 2 text limits it to 32 list-members; if adding a member would exceed that, the rightmost is removed. Keys start with a lowercase letter or digit and may contain up to 255 further characters from a-z, 0-9, _, -, *, / and @; values are up to 256 printable ASCII characters excluding comma and equals. Only one entry per key is allowed, and a service that updates its entry moves it to the left of the list, so the most recently active vendor appears first.

Size matters because proxies enforce header budgets. The spec asks implementations to propagate at least 512 characters of tracestate where possible, and when truncating, to remove whole entries, dropping those longer than 128 characters first. Two rules protect traceparent: if traceparent fails to parse, tracestate is not parsed either, and a tracestate without a traceparent must be discarded. A malformed tracestate never invalidates traceparent.

What a service may and may not change

The Level 2 text restricts mutation tightly. A service that participates in a trace may update the parent ID, may update the sampled flag (which requires a new parent ID), may restart the trace, or may downgrade the version. Anything else is a violation. A restart regenerates trace ID, parent ID and flags, and is typically used at secure network entry points to limit denial-of-service exposure; on restart, tracestate should be cleared unless preserving it is a conscious decision. This restart option is the foundation of the trust-boundary pattern discussed later.

How OpenTelemetry Injects and Extracts

OpenTelemetry defines propagation with a deliberately small surface. A TextMapPropagator has Inject(context, carrier, setter) and Extract(context, carrier, getter), plus a Fields method listing the header names it writes. The carrier is whatever transport object you have, and the getter and setter are tiny adapters that teach the propagator how to read and write it. That indirection is why the same W3C propagator works for HTTP headers, gRPC metadata and Kafka record headers without knowing anything about any of them.

Several rules in the spec shape real-world behaviour. Extract must not throw on unparseable input and must not overwrite a valid value that is already in the supplied context, so a garbage header degrades to “no remote parent” rather than crashing the request. The key type is restricted to US-ASCII and valid HTTP header names. For HTTP carriers, Get must be case-insensitive, which matters because HTTP/2 lowercases every header while HTTP/1.1 intermediaries sometimes capitalise them. Reusing a carrier object across requests without clearing the propagator’s Fields first can leak a previous request’s identity into the next, a subtle bug in pooled-client code.

Composite propagators and the default set

Real fleets are not homogeneous, so OTel lets you compose propagators. A composite runs its members in specified order on both inject and extract. On inject, a composite of tracecontext, baggage and b3 would write all three header families, which lets a migrating fleet speak W3C to new services and B3 to legacy ones simultaneously. On extract, the first propagator that finds a valid context in the carrier wins for the span context, while baggage extraction is independent.

Most SDKs enable W3C Trace Context and W3C Baggage by default and let you override them with the OTEL_PROPAGATORS environment variable, a comma-separated list. The values I would expect to see (tracecontext, baggage, b3, b3multi, jaeger, xray, ottrace, none) are supported to varying degrees by language SDK and extension package, so verify against your SDK’s documentation before relying on a particular one; b3 and jaeger in particular are often separate packages rather than part of the core SDK. Setting OTEL_PROPAGATORS=none is occasionally useful in tests and occasionally catastrophic in production, because it quietly disables propagation without any error.

A minimal Python example

The following Python shows manual injection and extraction, which is what instrumentation libraries do for you. It uses the standard OTel API calls.

from opentelemetry import trace, propagate, context
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter

provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("demo")

# --- caller side ---
with tracer.start_as_current_span("client-call"):
    headers = {}
    propagate.inject(headers)          # writes traceparent (and baggage if set)
    print(headers)                     # {'traceparent': '00-...-01'}

# --- callee side ---
def handle(request_headers):
    ctx = propagate.extract(request_headers)   # new Context holding the remote parent
    with tracer.start_as_current_span("server-handler", context=ctx) as span:
        # span.parent is the caller's span id; trace id is shared
        do_work()

def do_work():
    # child spans created here inherit via the ambient context
    with tracer.start_as_current_span("db-query"):
        pass

handle(headers)

Notice that extract returns a new context rather than mutating the current one. Nothing is active until you pass it to start_as_current_span or attach it. Forgetting that step is the commonest hand-written-instrumentation mistake: the code extracts a perfectly good context, never uses it, and the server span becomes a new root.

The same thing in Go

Go makes the ambient context explicit because it threads context.Context through every call. That removes a class of async bugs and replaces it with a different one: forgetting to pass the context.

import (
    "context"
    "net/http"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/propagation"
)

func init() {
    otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
        propagation.TraceContext{}, propagation.Baggage{},
    ))
}

func callDownstream(ctx context.Context, url string) error {
    req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
    otel.GetTextMapPropagator().Inject(ctx, propagation.HeaderCarrier(req.Header))
    _, err := http.DefaultClient.Do(req)
    return err
}

func handler(w http.ResponseWriter, r *http.Request) {
    ctx := otel.GetTextMapPropagator().Extract(r.Context(),
        propagation.HeaderCarrier(r.Header))
    tr := otel.Tracer("svc")
    ctx, span := tr.Start(ctx, "handle")
    defer span.End()
    _ = callDownstream(ctx, "http://svc-b/work") // pass ctx, not r.Context()
}

The comment on the last line is the whole lesson. Calling callDownstream(r.Context(), ...) would compile, run and silently inject the pre-span context, so the downstream parent would be the caller’s span rather than this handler’s span, flattening the tree by one level.

In-Process Propagation: Where Context Actually Gets Lost

Most broken traces are not network problems. The header arrived fine; the code simply lost the ambient context between extraction and the outbound call. Understanding why requires knowing how each runtime stores “the current context”.

Python uses contextvars, which are copied into asyncio tasks at creation, so await chains generally preserve context, but plain threading.Thread does not copy it and concurrent.futures thread pools run the callable in a worker whose context is whatever that worker last had. Java traditionally used thread-locals, which do not follow work onto a pooled thread, so the OTel Java agent wraps executors and common async frameworks to capture the context at submission and restore it on execution. Node.js uses AsyncLocalStorage (or the older async hooks), which follows promises and callbacks but can be broken by libraries that use their own queues or pre-bound callbacks. Go has no ambient context at all, so propagation is exactly as good as your discipline in passing ctx.

Figure 3: Kafka propagation with a batch consumer. Each record carries its own creation context; the batch Process span links to every one of them instead of choosing a single parent.

Thread pools and executors

The failure pattern is identical across languages. Request handling happens on thread T1 with the context attached. The code submits a task to an executor, a worker thread T2 picks it up, and T2 has no context. Any span started there becomes a root, and any outbound HTTP call from T2 injects a brand-new trace ID. The fix is to capture the context at submit time and attach it at run time. In Java, the OTel API provides Context.taskWrapping(executor) and Context.wrap(runnable); in Python, contextvars.copy_context().run(fn); in Node, wrap with AsyncResource.bind. Agents and auto-instrumentation handle the common executors, but a custom thread pool or a vendor-specific scheduler is exactly where the automatic wrapping tends not to reach.

Goroutines, reactive streams and callbacks

In Go, spawning go func() { ... }() inside a handler and using the outer ctx works, but there is a trap: if the goroutine outlives the request, the request context is cancelled when the handler returns, and every call made with it fails with context canceled. The correct pattern for fire-and-forget work is to start a new background context and carry only the trace link, not the cancellation. In reactive Java (Project Reactor, RxJava), operators run on arbitrary schedulers and the thread-local approach fails unless the library integrates with the reactive context mechanism. In Node, callback-style libraries that queue work on an internal pool can break AsyncLocalStorage propagation. In each case the symptom is the same: a span appears as a new root, or an outbound call starts a new trace, immediately after an asynchronous boundary.

A worked example of silent fragmentation

Take a checkout service that calls inventory, pricing and fraud checks in parallel. Suppose it uses a custom thread pool with eight workers and that the pool is not instrumented. Each parallel call runs on a worker with no context. In illustrative numbers, if the pool services 40 requests per second and each makes 3 parallel calls, that is 120 outbound calls per second, each of which starts a fresh trace. Your backend now shows 40 incomplete checkout traces and 120 single-span orphan traces per second. Cost goes up (more traces to store), usefulness goes down (nobody can follow a slow checkout to its slow dependency), and the dashboards report healthy-looking per-service latency because each orphan is individually fast. This is why trace completeness belongs on the same dashboard as latency: orphan rate is a first-class health metric for the tracing system itself.

Crossing Process Boundaries: HTTP, gRPC and Message Brokers

Once context is safely in process, the next question is the carrier. The mechanics are the same everywhere, but each transport has quirks that decide whether the trace survives.

HTTP and gRPC

For HTTP, the carrier is the request header map and everything is straightforward until a proxy gets involved. Reverse proxies and API gateways generally forward unknown headers, but some strip headers over a size limit, and some security configurations use an allow-list of forwarded headers that predates traceparent. Browsers add a wrinkle of their own: a cross-origin request from front-end code will only carry traceparent or baggage if the server’s CORS policy lists them in Access-Control-Allow-Headers, otherwise the preflight fails and the application breaks. The Baggage spec calls this out explicitly. Treat that as an engineering decision, not a default: enabling browser-originated trace headers means every external page that can reach your API can try to inject trace identity.

gRPC carries propagation data in metadata, which maps to HTTP/2 headers. Keys are lowercased on the wire, so traceparent and baggage appear exactly as written, and gRPC interceptors on both client and server do the inject and extract. The usual failure is configuration, not protocol: a client interceptor is registered on one channel but not the one used for a retry path, or a server framework registers its interceptor after an authentication interceptor that short-circuits early and so never creates a span. Streaming RPCs add a subtlety: the context is established when the stream starts, so every message on a long-lived stream shares one parent, and per-message spans must be created deliberately.

Message brokers: Kafka headers, SQS and Pub/Sub attributes

Queues invert the timing: the producer and consumer are not alive at the same time, so the context must be stored in the message. Kafka (0.11 and later) supports record headers, which are ordered key-byte-array pairs that travel with the record. A producer instrumentation injects traceparent (and baggage where wanted) into those headers; the consumer extracts from them. The key observation is that the context is attached to the message at creation, not at delivery: whatever happens in between, including a retention period measured in hours or a consumer outage, does not change the creation context. For SQS the equivalent is message attributes, and for Google Pub/Sub it is message attributes too; each has its own limits on attribute count and size, so verify against your service’s current quotas before adding baggage-sized payloads.

Two details matter in practice. First, intermediaries must not rewrite the carrier. A stream-processing job that reads from topic A and writes to topic B must copy or regenerate the headers deliberately; many frameworks drop headers by default when they rebuild a record, which silently ends the trace at that hop. Second, the OTel messaging conventions say the producer should attach a creation context to each message and, if possible, in a way intermediaries cannot change.

Span links versus parent-child for consumers

The messaging semantic conventions make a clear choice that surprises many people: links are the default, parent-child is opt-in. The specification gives three reasons. Links are the only structure that works across all messaging models, a span can have only one parent so batches cannot be parented, and links work when consumption happens inside another span such as an HTTP server span. A Process span may use the creation context as its parent, but only in single-message scenarios, and instrumentations should document whether they do.

Figure 3 shows why. A batch consumer polls two records created by different producer spans, and the processing span must relate to both. A parent can name only one, so the honest model is two links. The trade-off is visual: many backends render links less prominently than parent-child edges, so a trace that is correct under the conventions may look disconnected in a waterfall view. Teams that want a single waterfall for single-message flows can opt in to parent-child in the instrumentation, accepting that it will not generalise to batch consumption.

Scenario	Recommended relation	Reason
Single message, push-based handler	Parent-child (opt-in) or link	One creation context maps cleanly to one parent
Batch receive, N messages	One link per message	A span has exactly one parent
Processing triggered inside an HTTP server span	Link, with ambient parent kept	Avoids re-parenting an existing span
Fan-out: one message, many consumers	Link from each consumer span	Consumers are independent traces by design
Retry or dead-letter redelivery	New span, link to original creation context	Preserves the origin across redelivery

The attributes that make these spans searchable come from the same conventions: messaging.system, messaging.destination.name, messaging.operation.type, messaging.batch.message_count and messaging.message.id among others. Using them consistently lets you query “all spans that processed messages from topic X” without relying on the link graph. For the delivery-semantics side of the same pipelines, including what happens when a record is redelivered, see Kafka exactly-once semantics, transactions and the idempotent producer, since a replayed record carries its original creation context and will appear as a second consumer span linked to the same producer.

A Kafka example with span links

from opentelemetry import trace, propagate
from opentelemetry.trace import Link

tracer = trace.get_tracer("orders")

class HeaderGetter:
    def get(self, carrier, key):
        for k, v in carrier:                # Kafka headers: list of (str, bytes)
            if k == key:
                return [v.decode()]
        return []
    def keys(self, carrier):
        return [k for k, _ in carrier]

def produce(producer, topic, payload):
    with tracer.start_as_current_span("orders publish", kind=trace.SpanKind.PRODUCER):
        carrier = {}
        propagate.inject(carrier)
        headers = [(k, v.encode()) for k, v in carrier.items()]
        producer.send(topic, value=payload, headers=headers)

def process_batch(records):
    links = []
    for rec in records:
        ctx = propagate.extract(rec.headers, getter=HeaderGetter())
        sc = trace.get_current_span(ctx).get_span_context()
        if sc.is_valid:
            links.append(Link(sc))
    with tracer.start_as_current_span("orders process", kind=trace.SpanKind.CONSUMER,
                                      links=links,
                                      attributes={"messaging.batch.message_count": len(records)}):
        for rec in records:
            handle(rec)

The consumer creates one span with N links and does not adopt any producer as parent. A reader of the trace can still jump from the processing span to each originating request, and sampling decisions made at the producers remain visible in the linked contexts.

Baggage: Application Data That Travels With the Request

traceparent identifies a request. baggage carries information about it: a tenant ID, a feature-flag cohort, a synthetic-traffic marker, a customer tier. Because baggage rides in the same context as the span and propagates automatically, any downstream service can read it without every API in the chain adding a parameter. That is its entire value proposition, and also its main danger.

Format and limits

The header is named baggage, lowercase recommended, and its value is a comma-separated list of key=value members. Keys are RFC 7230 tokens, values use percent-encoding outside the permitted character set (and the percent sign itself must be encoded), and each member may carry optional properties after a semicolon, such as ;ttl=1, to which the spec assigns no built-in meaning. For example:

baggage: tenant=acme,tier=gold;origin=edge,synthetic=true

The Baggage specification requires platforms to propagate all members as long as the combined result has 64 or fewer members and is 8192 bytes or less. Beyond either limit a platform may drop members in any order it chooses, but it must not propagate partial members. The limits apply to all baggage headers combined. A proxy with a lower header-size ceiling may reject the request outright before these limits matter, which is why a small, disciplined baggage set is wise.

Baggage is not span attributes

A frequent misunderstanding is that putting a value into baggage makes it appear on spans. It does not. Baggage is context; span attributes are data on a specific span. To record a baggage value on spans, you must copy it, either in code or with a span processor that reads the baggage and stamps an attribute on every new span. A baggage-to-attribute span processor is a common pattern for tenant-level analysis, with an allow-list of keys so that arbitrary baggage cannot become unbounded attribute cardinality.

from opentelemetry import baggage, context
from opentelemetry.sdk.trace import SpanProcessor

ALLOWED = {"tenant", "tier"}

class BaggageToAttributes(SpanProcessor):
    def on_start(self, span, parent_context=None):
        for k, v in baggage.get_all(parent_context).items():
            if k in ALLOWED:
                span.set_attribute(f"app.{k}", str(v))

# set baggage at the edge after authentication
ctx = baggage.set_baggage("tenant", "acme")
token = context.attach(ctx)

The cost model

Baggage costs bytes on every hop. Illustrative arithmetic: a 300-byte baggage header on a request that fans out to 12 internal calls adds 3.6 KB of header traffic per user request, and at 5,000 requests per second that is about 18 MB per second of additional header bytes inside the cluster, before accounting for retries. That is rarely a bandwidth problem and often a CPU problem: every hop parses and re-serialises the header, and every sidecar or gateway that inspects it adds latency. The bigger cost is cognitive. Baggage is an implicit global variable for the whole fleet, and implicit globals are hard to reason about, version and deprecate.

Trust Boundaries: Who Gets to Set the Trace?

Trace headers are untrusted input the moment they cross a boundary you do not control. The W3C security considerations are clear on the risks: naively continuing any inbound trace with the sampled flag set can overload a service, enable trace ID collisions and inflate tracing costs; header length and content must be validated to prevent buffer overflows and injection; and tracestate can reveal the caller’s monitoring software. The Baggage spec adds that it may carry sensitive data and that systems should either keep confidential information out of it or prevent it from crossing trust boundaries.

The edge pattern: restart, then link

The spec’s own mitigation is the restart. At the internet-facing edge, discard inbound traceparent, tracestate and baggage from untrusted callers, generate a fresh trace, and optionally record the inbound trace ID as a span link or attribute so that a partner who sent a trace ID can still correlate. This stops an attacker from forcing sampled traces, from choosing your trace IDs, and from smuggling baggage into internal services that trust it. For authenticated partners with whom you have agreed to share traces, make continuation conditional on authentication and rate limit it.

Conversely, outbound headers leak too. Calling a third-party API with your baggage header attached sends your tenant IDs to a vendor. OTel instrumentation for HTTP clients usually injects on every request unless told otherwise, so the safe default for external hosts is an allow-list: propagate to internal domains, strip to everything else. Response headers deserve the same caution: the spec notes that response headers can leak to cross-origin callers and should be included only for systems that participated in the trace.

Concrete hardening checklist at the gateway

Strip inbound baggage unless the caller is authenticated and the keys are on an allow-list.
Enforce a maximum header length before parsing and reject oversized values.
Regenerate trace ID for unauthenticated traffic, and clear tracestate on restart.
Rate limit trace continuation per client so a flood of sampled flags cannot become a cost attack.
Remove or encode personal data from baggage values; never put emails, tokens or session IDs in baggage.
Inject only to hosts on an internal allow-list for outbound calls.

Legacy Formats: B3, Jaeger and Migration

Real fleets include services that predate W3C support. Zipkin’s B3 comes in two encodings. The multi-header form uses X-B3-TraceId (32 or 16 lowercase hex characters), X-B3-SpanId, an optional X-B3-ParentSpanId, X-B3-Sampled (1 or 0) and X-B3-Flags for debug. The single-header form is b3={TraceId}-{SpanId}-{SamplingState}-{ParentSpanId}, where only the first two are required; sampling state may be 1, 0, d for debug, or omitted to defer. The Jaeger native format uses an uber-trace-id header carrying trace ID, span ID, parent span ID and flags in a single colon-separated value.

Differences matter during migration. B3 allows 64-bit trace IDs, so a W3C propagator must left-pad them to 128 bits, which interacts with the Level 2 random-flag rule: a left-padded legacy ID may not meet the randomness constraint, and the flag must then be zero. B3 has a first-class “defer” sampling state and a debug flag with no direct W3C equivalent, so round-tripping through W3C can lose them. And B3 carries the parent span ID explicitly while W3C does not, so the shapes of the trees can differ subtly.

The migration recipe that works is dual propagation. Configure every service with a composite propagator that injects both W3C and the legacy format and extracts either, ordered so W3C wins when both exist. Migrate in waves, watch the orphan-trace rate, and remove the legacy propagator only when metrics show that no inbound traffic uses it. Never switch a gateway to W3C-only before its callees can extract W3C; the failure is silent and arrives as a gradual rise in single-span traces.

Sampling Decisions and the Header

The sampled flag converts trace propagation into a cost lever. Under head sampling, the root decides, usually with a probabilistic sampler on the trace ID, and every downstream service follows the flag. OTel’s parent-based sampler formalises this: if there is a remote parent, follow its sampled flag; otherwise apply the configured root sampler. The result is complete traces or none, with a cost that scales with the sampling ratio.

Two propagation hazards undermine this. First, a service that does not propagate context at all starts a new root, and with a 10 percent probabilistic root sampler that new root is kept independently of its upstream, so the trace is both fragmented and rebiased. Second, a hop that mutates the sampled bit without generating a new parent ID violates the mutation rules and makes the backend’s reconstruction ambiguous. The random-trace-ID flag matters here: probabilistic samplers that hash the trace ID depend on that ID being uniformly random, and when a legacy system produces structured or sequential IDs, the sampled fraction can differ materially from the configured ratio. If you are choosing between head and tail strategies, the discussion in tail-based sampling with the OpenTelemetry Collector explains why tail sampling needs all spans of a trace to reach one collector instance, a requirement that only holds if propagation produced a single trace ID in the first place.

Service Meshes, Gateways and Edge Networks

Meshes and gateways are the places where context is simultaneously most useful and most fragile. An Envoy-based sidecar can create spans for the network hop itself and will forward and generate trace headers, but it cannot see inside your application: when the application receives a request and makes an outbound call without copying the incoming trace headers, the sidecar sees two unrelated requests and starts two traces. This is the central lesson of mesh tracing: the sidecar can propagate headers it sees, but only the application can correlate an inbound request with the outbound requests it causes. Applications must propagate (or use an SDK that does) even when a mesh is present.

Gateways have the opposite problem: they may add or drop headers by policy. Check that your ingress controller passes traceparent and baggage, that any WAF or header allow-list includes them, and that the gateway’s own tracing integration uses the W3C format rather than a vendor default. For fleets that connect sites and devices over overlay networks, such as the setups in WireGuard vs Tailscale vs NetBird vs Headscale for edge fleet networking, the overlay itself is transparent to HTTP headers, so tracing breaks are usually in the application or gateway layers rather than the tunnel; the exception is a TLS-terminating or header-rewriting hop placed on the path.

Debugging a Broken Trace

When a trace splits, resist the urge to inspect the backend first. A broken trace is a statement about three places: the wire, the extraction and the ambient context. Figure 4 turns that into a decision procedure.

Figure 4: A debugging tree for fragmented traces. Establish whether the header exists on the wire, whether the callee extracted it, and whether context was still active at the outbound call.

Start at the hop where the trace breaks. Capture the actual request on the wire, either with a debug log of outgoing headers, a proxy capture or curl -v against the endpoint, and look for traceparent. If it is absent, the caller never injected: no propagator was set, the HTTP client is not instrumented, or the request was built outside the traced call. If it is present on the wire but the callee’s span is a root, the callee failed to extract it: a proxy between them rewrote or stripped it, the callee uses a different propagator set (for example B3 only), or the framework’s instrumentation is disabled for that route. If the callee’s server span is correct but its own outbound calls are orphans, context was lost inside the callee, the executor and async problems from earlier.

Useful diagnostics

Log the traceparent header at the gateway and at each service entry; matching trace IDs across logs show exactly where identity changes.
Compare trace IDs at the break: same ID with a missing parent means the parent span was never exported (a sampling or export problem), whereas a different ID means propagation failed.
Count root spans per service. A service that should never be a trace root but has a high root count is dropping context.
Inject trace IDs into log lines so logs and traces correlate even when the trace tree is imperfect.
Test with a synthetic request that carries a known traceparent and assert that every hop echoes the same trace ID.

One failure deserves separate mention: the same trace ID with parent spans missing is frequently mistaken for a propagation bug when it is actually a data-loss bug. If the backend shows spans whose parent ID refers to a span that never arrived, a collector queue overflowed, an exporter timed out, or a tail sampler dropped one side. Propagation worked; delivery did not.

Trade-offs, Gotchas, and What Goes Wrong

Header size and proxies. Every propagated byte is paid on every hop. A traceparent is a fixed 55 characters at version 00, which is cheap, but tracestate and baggage can grow, and a single hop with a lower header limit can reject traffic. The failure appears as sporadic 431 or 400 responses on only the heaviest requests, which is a nasty pattern to diagnose because the correlation is with header size, not endpoint.

Cardinality and cost. Allowing arbitrary baggage keys to become span attributes lets a single unbounded value, a user ID or request ID, explode attribute cardinality in the backend. Use allow-lists.

Trace ID reuse and long-running workflows. A trace that spans a batch job, a human approval and a nightly settlement has a lifetime in days. Many backends bound trace duration or assembly windows, so such flows are better modelled as separate traces connected by links, each short enough to assemble.

Over-trusting the sampled flag. If you honour inbound sampled flags from the internet, an attacker can inflate your tracing bill and your collector load by sending the flag on every request. Restart at the edge and rate limit.

Version drift. The version field exists so the format can evolve. A conforming parser handles higher versions by parsing the known leading fields and emitting version 00, and restarts when the header is too short. Libraries that hard-reject any non-00 version will break when a newer peer joins.

Opinion: baggage is overused. My view, which is a judgement and not a standard, is that baggage should be rare and boring: a handful of low-cardinality, non-sensitive routing or tenancy hints, set once at the edge, read-only downstream. When a team reaches for baggage to pass business data between services, it is usually a sign that an explicit API field would be clearer, versioned and auditable.

Practical Recommendations

Treat propagation as infrastructure with its own tests, owners and metrics, not as a side effect of installing an agent. Set the propagators explicitly in configuration so that behaviour does not depend on defaults, and put the same setting in every service template. Standardise on W3C Trace Context and Baggage for new work, use dual propagation to bridge legacy systems, and record a date by which the legacy format is removed.

Decide trust policy once, at the edge, and encode it in the gateway: restart for unauthenticated traffic, allow-list baggage keys, cap header sizes and rate limit trace continuation. Decide the messaging model once as well: links by default, parent-child only for documented single-message flows, and creation context copied explicitly through any stream processor that rebuilds records.

Finally, measure the tracing system itself. Track the fraction of root spans per service, the ratio of linked to unlinked consumer spans, the share of traces with a missing parent, and the count of requests carrying traceparent at each tier. Those four numbers surface almost every propagation regression within a day of a deploy.

Set OTEL_PROPAGATORS (or the SDK equivalent) explicitly in service templates.
Wrap every custom executor, scheduler and async boundary with context capture and restore.
Pass ctx through every Go call chain; lint for context.Background() inside handlers.
Copy headers deliberately in Kafka Streams, Flink and any consumer-producer relay.
Use span links for batch consumers; document any opt-in parent-child.
Strip or allow-list baggage at trust boundaries, inbound and outbound.
Keep baggage under a few hundred bytes and never include personal data.
Run a synthetic known-traceparent canary through every tier on each release.
Alert on root-span rate for services that should never be roots.

Frequently Asked Questions

What is distributed tracing context propagation?

It is the mechanism by which trace identity travels with a request across process, thread and network boundaries so that independently reported spans can be assembled into one trace. Across the network it uses headers such as W3C traceparent, and inside a process it uses an ambient context object. Every instrumented hop extracts the incoming context, makes it current, and injects an updated copy into each outgoing call, replacing the parent ID with its own span.

What is the difference between traceparent, tracestate and baggage?

traceparent is the mandatory, fixed-format identity: version, trace ID, parent span ID and flags. tracestate is an optional, vendor-specific list of key-value members, limited to 32 entries, that lets tracing systems carry their own data alongside the identity. baggage is a separate W3C header for application-defined key-value pairs with its own limits, and it is independent of tracing. Only traceparent is required to link spans into a trace.

Should I use span links or parent-child for Kafka consumers?

OpenTelemetry messaging conventions make links the default because a span has only one parent, which makes batches impossible to parent, and because links work across all messaging models. Parent-child is an opt-in for single-message processing, and instrumentations should document it. For batch consumers, create one processing span with a link to each message’s creation context. Single-message flows can opt in to parent-child if a unified waterfall is worth the loss of generality.

Why do my traces break at async boundaries?

Because the ambient context is stored in a mechanism tied to the execution unit, such as a thread-local, contextvars or AsyncLocalStorage, and handing work to another thread, pool worker, goroutine or queue drops it unless something copies it. The fix is to capture the context where work is submitted and attach it where work runs. Auto-instrumentation covers common executors but not custom schedulers, which is where most orphan spans originate.

Is it safe to propagate baggage to third-party APIs?

Usually not by default. Baggage may hold identifiers or data you would not intend to share, and the W3C specification warns that it can carry sensitive information and should either exclude confidential data or be kept from crossing trust boundaries. HTTP client instrumentation tends to inject on every request, so configure an allow-list of internal hosts, and strip baggage on outbound calls to external domains.

Do I still need B3 or Jaeger propagation in 2026?

Only while legacy services or tools that cannot read W3C headers remain in the path. OpenTelemetry supports composite propagators, so you can inject both formats during migration and extract either, with W3C taking priority. Remove the legacy propagator when your metrics show no inbound traffic uses it. Note that conversions can lose B3-specific concepts such as the debug flag and the defer sampling state.