OTel Collector vs Vector vs Fluent Bit: 2026 Telemetry Pipeline

OTel Collector vs Vector vs Fluent Bit: 2026 Telemetry Pipeline

OTel Collector vs Vector vs Fluent Bit: 2026 Telemetry Pipeline

Every observability stack eventually hits the same fork. Something has to receive logs, metrics and traces at the edge, reshape them, and ship them to a backend without becoming the most expensive component in the cluster. In late 2026 that “something” is almost always one of three agents, and the otel collector vs vector vs fluent bit question quietly decides your telemetry bill, your per-node footprint and how much logic you can pull out of application code.

The three are not interchangeable. One is the reference implementation of a vendor-neutral protocol, one is a Rust data router built around a safe transformation language, and one is a small C engine that grew up on constrained hardware. Pick wrongly and you meet it as CPU throttling on edge nodes, silently dropped logs during a burst, or a pipeline you can only express with a zoo of sidecars.

This rewrite replaces older numbers with current versions and adds the failure behaviour that single-number benchmarks hide. You leave with a decision rule, a two-tier reference topology, and a checklist for buffering and delivery.

What this covers: where each agent came from, what changed by October 2026, a sourced benchmark and its caveats, VRL vs OTTL vs Lua, buffering and backpressure, topologies, cost and security, migration, and a decision matrix.

Context and Background

The three tools solve overlapping problems from different starting points, and their defaults betray their origins.

The OpenTelemetry Collector is the vendor-agnostic service of the OpenTelemetry project. Its documentation describes it as a way to receive, process and export telemetry that can replace multiple separate agents, with retries, batching, encryption and filtering of sensitive data handled for the application. It is built from receivers, processors, exporters, connectors and extensions, and the project itself calls its overall stability “mixed” because each component carries its own maturity label. The OTLP logs data model it carries is marked Stable in the specification.

Fluent Bit began in 2015 as a lightweight forwarder in the Fluentd family. The project marked its tenth anniversary with the v4.2 release in November 2025, and it ships inside the managed Kubernetes distributions of the large clouds. Its design centre is a small footprint written in C.

Vector is a Rust observability data router, stewarded by Datadog (a fact from the project’s public history; I did not re-verify governance terms for this post). It is built around VRL, the Vector Remap Language, and a graph of sources, transforms and sinks. Its release cadence is brisk: version 0.60.0 shipped on 19 October 2026, following 0.59.0 on 6 October and 0.58.0 on 26 August.

All three moved quickly in 2026, which matters because most benchmark and comparison articles, including the earlier version of this one, quote versions that are now several releases old:

Agent Version in the benchmark used below Latest seen October 2026
OpenTelemetry Collector v0.146.1 core v1.67.0 / collector v0.161.0
Vector v0.53.0 v0.60.0
Fluent Bit v4.2.3 v5.1.2 on the 5.x line, 4.2.8 backport on the 4.x line

Collector releases come roughly every two weeks and routinely carry breaking changes to Go APIs and feature gates, so pin versions. Fluent Bit 5.0 landed on 24 March 2026 with a shared HTTP server, worker support and, per the release announcement, OTLP receiving reported up to four times faster than v4.2. Anything you read about relative speed from before that is likely stale for Fluent Bit’s OTLP path.

For the Collector’s internals, read our OpenTelemetry Collector architecture and pipelines guide. For the broader edge stack choice, see the OpenTelemetry vs Prometheus and Loki edge observability ADR. This article is the three-way agent decision that sits between them.

The Core Argument: Pick by Where the Work Happens

The short answer: choose Fluent Bit when the node is small and shaping is light, Vector when transformation and routing dominate, and the OpenTelemetry Collector when you want one OTLP-native agent for logs, metrics and traces. Most mature fleets use two of them in a tier rather than crowning one.

My thesis, and it is an opinion, is that these agents should be compared by where in the pipeline the expensive work lives, not by a single logs-per-second figure. Collection is cheap and parallel. Parsing, redaction, enrichment, sampling and routing are expensive and want to be done once, reviewed in one place, and versioned. A tool that is excellent at the first job (Fluent Bit) and one that is excellent at the second (Vector) are not competing as much as the headline numbers suggest.

otel collector vs vector vs fluent bit positioned by design centre

Figure 1: The three agents share one pipeline shape but differ in design centre. Fluent Bit minimises footprint, Vector maximises transformation depth, and the Collector maximises native OTLP signal coverage.

Figure 1 shows the shared shape. Every agent receives, parses, transforms, buffers and exports. They differ on how much of the work you can comfortably do in the agent, and on how many signal types are first-class.

Signal coverage is the first filter

The Collector treats logs, metrics and traces as equal citizens of the OTLP data model. If your instrumentation already emits OTLP from SDKs, the Collector is the lowest-friction receiver, and the OpenTelemetry SDKs assume a local Collector endpoint by default.

Vector’s OpenTelemetry source exposes three outputs, logs, metrics and traces, listening on the standard OTLP ports 4317 for gRPC and 4318 for HTTP. But the documentation labels the component beta, with metrics and trace support experimental. It also notes that OTLP-preserved metrics are emitted as log events, so Vector’s native metric transforms such as aggregation cannot process them. That is a concrete reason not to make Vector your trace path yet.

Fluent Bit 5.0 refactored its HTTP-based inputs, including the OpenTelemetry input, onto one shared server with worker support, and widened OTLP compatibility across logs, metrics and profiles, with stronger metrics support including exponential histograms. Traces are handled, but the project’s centre of gravity is still logs.

Language runtime shapes the failure profile

Fluent Bit is C. Vector is Rust, with no garbage collector. The Collector is Go, with a GC. This is not a style point. A Go agent under memory pressure has pauses and a larger resident set per unit of throughput, which the benchmark below shows. A C agent has the tightest footprint but a history of memory-safety bugs; the Fluent Bit release notes for 5.0.7 list “many out-of-bounds read fixes across input plugins.” Rust gives Vector memory safety in safe code, which is why it is attractive for internet-facing receivers.

Ownership and governance differ

The Collector and OTLP live under the OpenTelemetry project in the CNCF, with many vendors contributing and a public stability policy. Fluent Bit sits in the Fluent community, whose members proposed in late 2025 a unified Fluent organisation within the CNCF with no change to licences or names. Vector is vendor-led. None of this is disqualifying. It is a risk you price in: a vendor-led project can change direction faster than a foundation one, and a foundation project can move slower on your pet feature.

Benchmark Evidence and How Far to Trust It

The most useful independent comparison available in 2026 is the VictoriaMetrics log-collector benchmark, and it is worth reading both for its numbers and for its candour about its own limits.

Methodology

Per the write-up, every collector ran from its official Helm chart on a single-node kind cluster on a Google Cloud n2-highcpu-32 VM (32 vCPUs, 32 GiB RAM), with limits of one CPU core and 1 GiB of memory each and no tuning. Log generators wrote JSON to stdout at about 216 bytes per record, each with a monotonically increasing sequence ID so a verifier could detect gaps. Scenarios ran 1, 10, 50, 100, 150 and 200 pods, and the 100-pod case was the primary comparison.

Results

Collector (benchmarked version) Max logs/sec, 100 pods CPU cores at ~10k logs/s Mean memory at ~10k logs/s
Fluent Bit v4.2.3 31,300 0.260 78.10 MiB
Vector v0.53.0 25,000 0.412 153.50 MiB
OpenTelemetry Collector v0.146.1 20,500 0.491 106.83 MiB

Source: VictoriaMetrics, March 2026. VictoriaMetrics’ own agent, vlagent, topped the table at 143,000 logs/sec, so the three agents here ranked second to fourth in that field. Grafana Alloy, Promtail, Filebeat and Fluentd all trailed.

The ordering matches each design: Fluent Bit leads on throughput and CPU, Vector sits in the middle, and the Collector pays for its generality. The CPU gap is the part I would carry into capacity planning. At 10,000 logs per second, the Collector used roughly 1.9 times the CPU of Fluent Bit, and Vector about 1.6 times.

What the headline numbers hide

The benchmark’s own conclusions section is more interesting than its throughput table, and the earlier version of this post missed it.

First, Fluent Bit and Vector sent split records during file rotation, even at low CPU. In an hour at 10,000 logs per second from two pods, Fluent Bit produced 34 malformed records and Vector 2. No other collector in the test produced any. If you parse JSON downstream, a truncated record is a parse failure or, worse, a silently wrong event.

Second, the memory column needs care. The write-up states that Fluent Bit and Filebeat exceeded the 1 GiB limit under load and were OOM-killed. The 78 MiB figure is the steady state at 10,000 logs per second, not a promise at saturation.

Third, Vector’s defaults can lose logs. With the default glob_minimum_cooldown_ms of 60 seconds, logs from newly created pod files could be dropped silently; the author lowered it to 10 seconds. Vector’s open file descriptors also grew without bound under high load, and delivered logs lost Kubernetes metadata when a pod was deleted with a backlog. Fluent Bit’s default of 10 seconds avoids the discovery problem.

Caveats you must apply

This is one benchmark, by an engineer at a company that sells a competing collector, as the article itself acknowledges. All collectors were untuned, on a single node, and the Collector run used a contrib-style configuration whose parsing work differs from the others. The versions are also now behind: Vector has moved from 0.53 to 0.60, the Collector from 0.146 to 0.161, and Fluent Bit onto a 5.x line with a re-architected worker model in 5.1.0. Treat the ratios as directional and rerun the published code on your own log shape, because JSON versus regex-parsed text, multiline joins and compression change results far more than the choice of agent.

Transformation Languages: VRL, OTTL and Lua

Where these agents truly diverge is in-flight data manipulation. This decides how much logic you can pull out of applications and how safely you can change it.

vrl vs ottl vs lua record flow in the otel collector vs vector vs fluent bit pipeline

Figure 2: A record flows from receive through parse and transform into a buffer, then to a batched, retried send. A slow backend pushes back through the buffer, which either pauses the input or drops data depending on configuration.

Figure 2 traces one record. The transform stage is where the languages differ, and the buffer stage is where failure behaviour differs.

VRL: safe by construction

Vector documents VRL as an expression-oriented language with two headline properties. It is fail-safe: a VRL program will not compile unless all errors thrown by fallible functions are handled. And it is checked at compile time for types, so passing an integer to parse_json! is an error before the pipeline starts. The documentation also states that VRL compiles to native Rust code with no separate runtime and no garbage collector.

transforms:
  clean:
    type: remap
    inputs: [kubernetes_logs]
    source: |
      . = parse_json!(.message)
      del(.user_email)
      .env = "prod"
      if .level == "debug" { abort }

That fail-safe rule is the real feature. In a pipeline that handles millions of events, a runtime exception in a transform is a production incident. VRL forces you to write parse_json! (abort on error) or handle the error explicitly, and refuses to deploy otherwise. For a regulated team that reviews pipeline changes, this is worth more than raw speed.

OTTL: native to the OpenTelemetry data model

OTTL, the OpenTelemetry Transformation Language, is a small domain-specific language whose statements pair a function with an optional condition. Its README documents separate contexts for resource, instrumentation scope, span, span event, metric, datapoint and log (profiles are in development), and lists beta status for traces, metrics and logs. It is used by the transform processor, the filter processor, the tail-sampling processor and the routing connector.

processors:
  transform/logs:
    log_statements:
      - context: log
        statements:
          - set(attributes["env"], "prod")
          - delete_key(attributes, "user_email")
          - set(severity_text, "ERROR") where severity_number >= 17

Two limits are worth knowing. OTTL does not support cross-signal interactions, so a statement cannot copy a log body into a span attribute. And because it is scoped to the OTLP model, you reason about resources, scopes and attributes, which is a strength if your data already is OTLP and a tax if it is raw text lines.

Lua and built-in filters in Fluent Bit

Fluent Bit combines built-in parsers and filters with Lua scripting for custom logic. Version 4.2 added content-aware routing in the Telemetry Router, which directs data by record metadata, attributes or tags instead of tags alone, and 5.0 added metric processors such as cumulative-to-delta. For a modest rule set that is plenty. For hundreds of conditional rules it turns into an unreviewable Lua block, and that is usually the signal to move the shaping upstream to a gateway running VRL or OTTL.

A quick comparison

Dimension OTel Collector Vector Fluent Bit
Transform language OTTL (beta for logs, metrics, traces) VRL Parsers, filters and Lua
Error handling Runtime, per statement Compile-time enforced Runtime
Data model OTLP Vector event, OTLP via source Chunked records, OTLP in and out
Cross-signal logic Not in OTTL, use connectors Limited Limited
Review ergonomics Statement lists Programs with tests Lua scripts
Runtime Go, GC Rust, no GC C

Vector also has a unit-test framework for VRL transforms, which pairs well with the compile-time checks. If you treat pipeline config as code with CI, that combination is the best of the three today.

Buffering, Backpressure and Delivery Guarantees

Throughput is irrelevant if the pipeline loses data on the one day the backend is down. Each agent exposes a different set of knobs, and the defaults decide what happens when you forget to configure them.

OpenTelemetry Collector

The exporter helper provides a sending queue and retry. Per the exporterhelper documentation, sending_queue is enabled by default with queue_size 1000 (counted in requests by default), num_consumers 10, and block_on_overflow false, meaning a request is rejected immediately when the queue is full. Data that cannot be queued is typically dropped and reported through the otelcol_exporter_enqueue_failed_* metrics. Retries default to an initial 5 seconds, a 30 second maximum interval, a 1.5 multiplier and a max_elapsed_time of 300 seconds, after which the batch is dropped unless you set it to 0.

For persistence, set sending_queue.storage to a storage extension. There is then no in-memory queue, the queue lives on disk, and items survive a restart. One catch from the docs: request context is preserved but auth extension context is not, so authentication information may be unavailable when persisted data is replayed. The practical consequence is that a five-minute default retry window means a backend outage longer than five minutes loses data unless you raised it.

Vector

Vector buffers per sink. The HTTP sink reference lists buffer.type as memory (default) or disk, with when_full of block (default) or drop_newest. Memory buffers hold 500 events by default and lose data on a crash. Disk buffers sync every 500 ms, require a minimum size of about 256 MB, and are slower but durable. The default block behaviour applies backpressure upstream, which is safe for pull-based sources like file tailing, where the file simply waits, and dangerous for push-based sources, where the sender may time out and drop.

Fluent Bit

Fluent Bit buffers per input. storage.type defaults to memory, with filesystem as a hybrid mode that writes chunks to disk through mmap and keeps a copy in memory when there is room, and memrb as a fixed-size ring that drops the oldest chunks when full. mem_buf_limit defaults to 0, meaning no limit, and a memory-only input pauses when it reaches the limit. With filesystem buffering, storage.max_chunks_up and storage.pause_on_chunks_overlimit control what happens next, and storage.total_limit_size on an output discards the oldest chunk in the queue when the cap is hit.

Note the sharp edge: mem_buf_limit defaults to unlimited. On a small node, an unbounded memory buffer during a backend outage is exactly how you get an OOM-killed agent, which is what the benchmark observed under load.

Choosing delivery semantics

Goal OTel Collector Vector Fluent Bit
Never lose on outage Persistent queue plus long retry Disk buffer plus block Filesystem storage plus total limit
Bounded memory, accept loss In-memory queue, small size Memory buffer plus drop_newest memrb or memory with mem_buf_limit
Back-pressure the source block_on_overflow true when_full block Pause input on limit

None of the three gives exactly-once delivery. Disk-backed retry is at-least-once and can duplicate records, so make downstream stores idempotent, for instance by keying on a record ID.

Deployment Topologies: Agent, Gateway and Sidecar

Placement often matters more than the agent. The Collector documentation lists agent, gateway and combined agent-to-gateway patterns, and the same three apply to every tool here.

A node agent (a DaemonSet in Kubernetes) runs once per host, tails local files and scrapes local endpoints. This is Fluent Bit’s home turf because its small footprint multiplies cleanly across a large fleet. A gateway is a smaller number of heavier instances that receive from all node agents, do expensive transformation once, and fan out to backends. This is where Vector and the Collector earn their keep. A sidecar pairs an agent with one workload, giving isolation at the worst footprint-per-workload ratio; reserve it for per-tenant security boundaries.

two tier telemetry pipeline for otel collector vs vector vs fluent bit

Figure 3: Thin agents on every node forward to a gateway tier that redacts, drops noise, samples, aggregates and routes by tenant and signal before the data reaches paid backends.

Figure 3 is the architecture most mature teams converge on. Economics drive it. Transformation is expensive, so doing it once in a gateway beats doing it on every node, and it concentrates the rules you most want to review, such as redaction and routing, in one versioned place. Keep node agents thin and dumb; keep intelligence in the gateway.

Wiring the tiers together

OTLP is the interoperability glue. Fluent Bit has an OpenTelemetry output and Vector has an OpenTelemetry sink, so a Fluent Bit DaemonSet can send OTLP to a gateway running either the Collector or Vector’s OpenTelemetry source. Fluent Bit 4.2 added AWS SigV4 authentication to its OpenTelemetry output and 5.0 added OAuth 2.0 client credentials to its HTTP output, which matters when your gateway sits behind an authenticating load balancer.

A minimal gateway Collector configuration shows the shape:

receivers:
  otlp:
    protocols:
      grpc: { endpoint: 0.0.0.0:4317 }
      http: { endpoint: 0.0.0.0:4318 }
processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 75
  transform/redact:
    log_statements:
      - context: log
        statements:
          - delete_key(attributes, "user_email")
exporters:
  otlphttp/loki:
    endpoint: https://loki.internal:3100/otlp
    sending_queue:
      enabled: true
      queue_size: 5000
service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [memory_limiter, transform/redact]
      exporters: [otlphttp/loki]

Put memory_limiter first in the processor chain so the gateway refuses new data before the kernel kills it. Recent releases also promoted the memory limiter extension to beta and added GC backoff and health events, which makes it worth revisiting older configs.

Capacity arithmetic for the gateway

A rough sizing exercise, using the benchmark ratios as directional inputs only. Suppose 400 nodes each produce 2,000 logs per second, or 800,000 per second fleet-wide. If a gateway replica sustains 15,000 logs per second after your transforms (a number you must measure, not assume), you need roughly 54 replicas, so provision around 70 for headroom and zone loss. The node agents cost 0.26 cores at 10,000 per second in the benchmark, so about 0.05 cores each at 2,000 per second if scaling is linear, which it will not quite be. The point is the structure of the calculation: gateway cost scales with transform complexity, node cost scales with volume.

Cost, Cardinality and Security

Beyond throughput, three concerns decide whether a pipeline is sustainable: what it costs, how it handles cardinality, and how it protects data.

Drop early, pay less

The biggest lever on telemetry cost is reducing volume as early as possible. Every byte dropped, sampled or aggregated before it crosses the network is a byte you do not transform, transmit and store. An agent that can drop debug logs and sample high-volume traces pays for itself. That is an argument for real transformation capability at least at the gateway. The Collector’s tail-sampling processor, which uses OTTL to choose spans, is the most mature option for traces because it can wait for a whole trace before deciding.

Cardinality control

Metrics cardinality is where observability bills explode. A label like user_id or request_id on a metric can multiply time series into the millions. Both the Collector, through processors, and Vector, through VRL, can drop or rewrite high-cardinality labels in flight. Decide which labels are queryable dimensions and strip the rest at the pipeline. Fluent Bit can do basic label manipulation but is less suited to complex surgery.

Security in transit and at rest

Telemetry carries request paths, identifiers and internal hostnames, so treat the pipeline as a data-handling system. All three support TLS. Fluent Bit 5.0 added mutual TLS verification on inputs via tls.verify_client_cert and OAuth 2.0 bearer-token validation on HTTP-based receivers, closing a gap that previously pushed teams towards a fronting proxy. Redact personal data with VRL or OTTL before it leaves your trust boundary, not after it has landed in a vendor’s backend.

Supply-chain hygiene is part of security too. The Collector’s ecosystem is large: the contrib distribution bundles hundreds of components, and the project’s builder lets you compile a custom binary with only what you need. A smaller binary is a smaller attack surface and a faster start, so build your own distribution for production gateways rather than shipping contrib wholesale.

Operating Each Agent in Production

Day-two operations reveal differences the feature tables do not.

Observability of the observer

You need metrics from the pipeline itself. The Collector exposes internal telemetry, including the otelcol_exporter_enqueue_failed_* counters that tell you data is being dropped at the queue, and Collector 1.58 added an in-flight requests metric for exporters. Fluent Bit 5.0 added output backpressure duration metrics and made fluentbit_hot_reloaded_times a counter. Alert on queue depth, enqueue failures and retry counts, not just process liveness. A pipeline that is up but dropping is worse than one that is down, because nobody pages.

Reloads and upgrades

Fluent Bit 4.2 introduced a Supervisor Mode that restarts child processes and a Hot Reload Watchdog that validates configuration changes at runtime. Release 5.1.1 fixed storage backlog handling on hot reload, a reminder that reload paths are where subtle bugs live. Collector releases through mid-2026 removed stabilised feature gates and renamed memory limiter metrics in version 1.61, so dashboards break across upgrades. Vector’s rapid 0.x cadence means minor versions can carry breaking changes before a 1.0. In all three cases, canary the upgrade on one node pool and diff the output volume before rolling the fleet.

Configuration as code

All three express a pipeline of receive, process and export. The Collector’s YAML wires receivers, processors and exporters into named pipelines per signal, which is verbose but explicit. Vector’s TOML or YAML defines a directed graph of sources, transforms and sinks, with VRL doing per-event work. Fluent Bit supports classic INI-style files and YAML. Whatever you pick, keep the config in version control, validate it in CI (Vector has vector validate and unit tests; the Collector has validate), and require review, because a bad routing rule can silently drop a signal.

Trade-offs, Gotchas and What Goes Wrong

The classic mistake is choosing an agent for a benchmark number and discovering the transformation model does not fit. Fluent Bit’s raw throughput is excellent, but if you then need heavy conditional routing you end up scripting large Lua blocks or bolting on a second tier anyway.

decision path for otel collector vs vector vs fluent bit

Figure 4: Start from the tightest constraint. Need OTLP traces natively, choose the Collector. Need heavy parsing and routing, choose Vector. Need a tiny edge footprint, choose Fluent Bit. When needs conflict, split into two tiers.

Figure 4 collapses the comparison into one question: which constraint dominates? When two conflict, the two-tier pattern is almost always the answer.

Specific failure modes

Rotation corruption. The benchmark found split records on rotation in Fluent Bit and Vector. Mitigate by using the container runtime’s log format with its partial-line flags, enabling multiline or reassembly features, and sampling downstream data for malformed JSON.

Silent discovery gaps. Vector’s 60 second default glob cooldown can drop logs from short-lived pods. Short jobs and CronJobs are exactly where you will notice. Lower the value and test with a pod that lives ten seconds.

Unbounded buffers on small nodes. Fluent Bit’s mem_buf_limit defaults to unlimited and Vector’s memory buffers default to 500 events. Neither default is a bounded-memory policy for your hardware. Set limits explicitly and rehearse the outage path.

Default retry windows. The Collector’s 300 second max_elapsed_time is shorter than many real outages. Decide your tolerance in hours, then size the persistent queue disk accordingly.

Signal scope creep. Teams adopt Fluent Bit or Vector for logs, then need traces and end up running the Collector anyway. If traces are coming, plan the two-tier split up front.

Version churn. Collector breaking changes arrive in most releases; Vector is pre-1.0; Fluent Bit moved from 4.x to 5.x in 2026. Pin, canary and read release notes. Verify the current maintenance status, licence and backend compatibility of whichever agent you standardise on before committing a fleet.

Where this analysis is weak

I have not run these agents myself for this rewrite; the throughput, CPU and memory numbers come from one third-party benchmark with a stated conflict of interest. Several newer versions were not benchmarked at all. Do your own run before committing.

Migrating From Legacy Collectors

Many teams arrive holding Fluentd, Logstash or Promtail. The benchmark is a reminder of what you are leaving: Fluentd and Filebeat began losing logs below 10,000 logs per second in that test, and Promtail reached 13,400 at its maximum.

Fluent Bit is the low-friction upgrade from Fluentd: same lineage, dramatically smaller footprint, familiar concepts. Teams leaving Logstash for JVM-footprint reasons frequently land on Vector, whose VRL covers most filter logic. Teams standardising on OpenTelemetry converge on the Collector so instrumentation, transport and collection share one data model. Promtail users should note it was surpassed in this benchmark by its successor lineage, Grafana Alloy, which is itself a Collector distribution; I did not verify Promtail’s current support status for this post.

Migrate incrementally. Run the new agent alongside the incumbent, tee a copy of traffic, and compare outputs at the backend before removing the old path. This catches parsing differences, timestamp quirks and dropped fields while the old pipeline still guarantees delivery. Because all three modern agents can emit and receive OTLP, you can point a legacy source at an OTLP-speaking gateway and migrate sources one at a time with no flag day. Resist porting every legacy transform verbatim; a migration is the best chance to delete rules nobody understands.

Decision Matrix

Workload Best first choice Why Watch out for
Constrained edge nodes, light parsing Fluent Bit Lowest CPU in the benchmark Unbounded mem_buf_limit, rotation splits
Heavy parsing, redaction, routing Vector VRL compile-time safety Default glob cooldown, 0.x breakage
Unified logs, metrics, traces, OTLP-native OTel Collector First-class signals Higher CPU, 5 minute retry default
Large fleet, mixed needs Fluent Bit on nodes, Vector or Collector gateway Transform once Two configs to govern
Trace tail sampling OTel Collector Tail-sampling processor Needs memory sized for trace windows
Regulated, reviewed pipeline changes Vector Fail-safe language plus unit tests Vendor-led governance

Practical Recommendations

Start from your tightest constraint. If it is edge footprint, begin with Fluent Bit and push heavy shaping to an aggregation tier. If it is transformation complexity, start with Vector. If it is vendor-neutral coverage of all three signals, start with the Collector and OTTL. Do not let a single throughput number override the transformation-model fit, and do not rely on a benchmark published before your target versions existed.

For anything beyond a small deployment, plan the two-tier architecture on day one. Use OTLP between tiers so you can swap either side later. Bound memory on every node, spill to disk, and rehearse a two-hour backend outage in staging.

  • Name your dominant constraint: footprint, transform complexity or signal breadth.
  • List the signals you need now and in six months, especially traces.
  • Pin versions and canary every upgrade; read the release notes for breaking changes.
  • Set explicit buffer limits and when_full or block_on_overflow policies.
  • Raise retry windows to match your real outage tolerance.
  • Test log rotation and short-lived pods with a loss detector such as a sequence ID.
  • Redact PII and drop noise at the gateway before data leaves your trust boundary.
  • Alert on enqueue failures and queue depth, not only process liveness.
  • Rerun the public benchmark on your own log shape before committing a fleet.

Frequently Asked Questions

Which is fastest, OTel Collector, Vector or Fluent Bit?

In the VictoriaMetrics benchmark under 1 CPU and 1 GiB, Fluent Bit v4.2.3 reached about 31,300 logs per second, Vector v0.53.0 about 25,000 and the OpenTelemetry Collector v0.146.1 about 20,500. That reflects design centres, with Fluent Bit’s C core built for efficiency. The author works for a competing vendor, the versions are now older, and parsing and codecs shift results, so treat it as directional and test your own workload.

When should I use Vector instead of Fluent Bit?

Choose Vector when transformation and routing dominate: complex parsing, conditional routing, enrichment and redaction. VRL is fail-safe, refusing to compile unless fallible functions are handled, which suits reviewed pipelines. Fluent Bit remains the better pick for a minimal footprint on constrained nodes with modest shaping. In the benchmark Vector used about 1.6 times Fluent Bit’s CPU at 10,000 logs per second, so budget for that on edge nodes.

Do I need the OpenTelemetry Collector if I already run Vector?

Only if you need OTLP-native traces or one vendor-neutral agent across all signals. Vector’s OpenTelemetry source is labelled beta, with metrics and traces experimental, and OTLP metrics arrive as log events that Vector’s native metric transforms cannot aggregate. Many teams keep Vector for log shaping and run a Collector gateway for traces and tail sampling, connected over OTLP.

What is the difference between VRL and OTTL?

VRL is Vector’s expression language, compiled to native Rust with compile-time type and error checking and no garbage collector. OTTL is the OpenTelemetry Transformation Language, used in the Collector’s transform, filter, tail-sampling and routing components, with contexts for resource, scope, span, metric, datapoint and log. OTTL is beta for traces, metrics and logs and cannot reference data across signals. Use VRL for heavy generic shaping and OTTL when you live in the OpenTelemetry model.

How should I handle backpressure and outages at the edge?

Bound memory and spill to disk. In Fluent Bit set storage.type filesystem with storage.total_limit_size; in Vector use a disk buffer with when_full set to block or drop_newest by intent; in the Collector enable a storage extension under sending_queue and raise the 300 second default retry window. Decide between never-drop and bounded-memory per signal, then rehearse the outage in staging before production.

Can I run all three together?

Yes, and many fleets effectively do. A tiny agent such as Fluent Bit sits on every node and forwards OTLP to a heavier Vector or Collector gateway that transforms once and routes onward. This gives edge efficiency and central expressiveness without forcing one agent to do everything. The cost is two configuration surfaces and two upgrade cadences, so only add a tier when a single agent cannot meet your constraints.

Further Reading

By Riju — about

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *