Time-Series Database Internals: InfluxDB 3, TimescaleDB, and QuestDB Under the Hood

Lede

As IoT deployments scale to millions of sensors, observability stacks ingest terabytes of metrics per day, and fintech platforms track tick-by-tick market data, the demands on data infrastructure have fractured the traditional database world. Relational databases (PostgreSQL, MySQL) and data warehouses (Snowflake, BigQuery) struggle with time-series workloads because they optimize for different access patterns: OLTP prioritizes random writes and point lookups; OLAP favors batch ingestion and complex joins.

Time-series databases (TSDBs) are purpose-built for a single, overwhelming use case: ingest metric data at high velocity, compress it aggressively, and query recent windows with sub-millisecond latency. But “purpose-built” masks enormous internal complexity. InfluxDB 3’s Apache Arrow + Parquet stack differs fundamentally from TimescaleDB’s Postgres-native hypertables and QuestDB’s memory-mapped columnar design. Each trade-off—write-ahead logs vs. in-memory buffering, chunk size vs. query latency, columnar compression vs. CPU overhead—ripples through performance, operational cost, and query correctness.

This post deconstructs three production TSDB architectures, exposing the algorithms and data structures that make them tick. You’ll learn why cardinality explodes performance, how continuous aggregates sidestep materialization costs, what makes Gorilla compression so effective, and when to pick each system for your infrastructure.

TL;DR

Time-series databases optimize for write-heavy, append-mostly workloads with time-windowed range queries, not random updates or complex joins.
InfluxDB 3 (IOx): Apache Arrow + Parquet columnar storage, DataFusion SQL engine, cloud-native horizontal scaling. Best for observability at scale.
TimescaleDB: Postgres extension with hypertables (time-partitioned tables), continuous aggregates, and transparent columnar compression. Best for users already in Postgres.
QuestDB: Memory-mapped column-oriented storage, out-of-order (O3) buffer, SIMD vectorized execution. Best for ultra-low-latency, high-cardinality financial data.
Compression techniques (Gorilla timestamp delta-of-delta + float XOR, ZSTD) reduce storage 10–100x, enabling weeks/months of hot data.
Cardinality (unique tag combinations) is the silent killer—exploding cardinality breaks indexing, bloats metadata, and tanks query performance. Enforce cardinality budgets.
Continuous aggregates avoid re-computing the same aggregations: pre-compute hourly/daily rollups incrementally as new data arrives.

Key Concepts
What Makes a Time-Series Database Different
InfluxDB 3 (IOx) Internals
TimescaleDB Internals
QuestDB Internals
Compression Techniques in Time-Series Data
Benchmarks & Feature Matrix
Edge Cases & Failure Modes
Implementation Guide: Choosing and Deploying a TSDB
FAQ
Where TSDBs Are Heading
References
Related Posts

Key Concepts

Before diving into architecture, establish the vocabulary of time-series systems.

Cardinality: The number of unique combinations of tag values in your dataset. If you have region=us|eu|asia (3 values) and host=host-001..host-999 (999 values), cardinality is 3 × 999 = 2,997. Cardinality explodes when a tag carries high entropy (e.g., request IDs, user UUIDs, API tokens). High cardinality is the primary killer of TSDB performance—each unique combination requires separate index entries, series metadata, and internal routing.

Series: A single time-series, uniquely identified by a metric name and tag set. Example: cpu_usage{region=us, host=host-001} is one series; adding a new host creates another.

Tags vs. Fields: Tags are indexed dimensions (low cardinality, used for filtering). Fields are measurement values (high dimensionality, not indexed). Example: metric temperature, tags location=building-A, field value=72.5. Tags enable fast filtering; fields are what you aggregate.

Downsampling: Reducing resolution over time by aggregating older data (e.g., store raw 1-second samples for 24 hours, 1-minute averages for 30 days, 1-hour for a year). Saves storage and query cost; trades precision for retention.

Continuous Aggregates: Pre-computed, incrementally-maintained rollups. Instead of re-computing the sum of all events in an hour, the system updates the aggregate as new data arrives. Critical for avoiding re-reads and re-computation during backfill.

Columnar Storage: Store all values of a single column together (age: [25, 31, 28, …]) rather than all fields of one row (row: [Alice, 25, NYC, …]). Columnar enables SIMD vectorization, dictionary encoding, and compression—essential for analytics. Trades random access for bulk scans.

Log-Structured Merge (LSM) Tree: A write-optimized data structure. New writes go to a fast in-memory buffer (memtable); when full, it’s flushed to disk as an immutable SSTable (Sorted String Table). Multiple SSTables are periodically merged (compacted) into larger files. LSM trades read amplification for write speed and sequential I/O.

Gorilla Compression: Facebook’s time-series compression algorithm (Pelkonen et al., 2015). Exploits temporal locality: timestamps differ by small deltas; float values change little between samples. XOR-ing floats and delta-of-delta encoding timestamps achieves ~1.3 bytes per sample.

What Makes a Time-Series Database Different

Time-series databases carve out a distinct operational space between traditional OLTP and OLAP systems.

Write-Heavy, Append-Only Workload: Relational databases assume a mix of reads and writes, with UPDATE and DELETE as first-class operations. TSDBs assume nearly all writes are append—you never update a sensor reading, only add new ones. This enables sequential writes to disk, bypassing random-access penalties that plague UPDATE-heavy databases. In-order append also simplifies concurrency control (no locking needed for future timestamps).

Time-Windowed Reads: Analytics queries almost always ask: “What was the average temperature in region=us over the last 7 days?” Not: “Find all rows where user_id = 42.” This enables aggressive time-based partitioning (chunks), index pruning (skip chunks before the query window), and early termination (stop scanning if you’ve reached the earliest timestamp).

High-Cardinality Dimensions: Observability generates labels naturally (request_id, span_id, trace_id, user_id, pod_name). Naïve indexing on all combinations explodes space and query planning complexity. Smart TSDB systems use inverted indexes (map tag values → series IDs) rather than traditional B-trees, and pre-filter before joining series.

Aggregate-First Semantics: OLAP queries compute SUM, COUNT, AVG over millions of rows. TSDBs assume this is the common case and optimize columnar storage and vectorized execution accordingly. Columnar layout (all values of one column contiguous) enables CPU SIMD instructions to process 8–16 values per cycle.

Lossy Compression is Acceptable: A data warehouse needs perfect precision; a time-series system can tolerate float32 instead of float64, lossy timestamp compression (rounding to nearest second), and downsampling. This trades accuracy for storage efficiency—storing a year of metrics in a few terabytes instead of petabytes.

InfluxDB 3 (IOx) Internals

InfluxDB 3 rewrote the engine as InfluxDB IOx, abandoning custom formats for industry-standard Apache Arrow (in-memory columnar), Parquet (on-disk columnar), and DataFusion (SQL query engine).

Write Path: Incoming writes (gRPC or HTTP) land in an Ingest layer, which buffers data in memory. When the buffer fills (or on timeout), it’s flushed to a Write-Ahead Log (WAL) for durability, then written as a Parquet file to cloud storage (S3, GCS, Azure Blob). WAL guarantees no data loss; Parquet is the persistent store.

Compute Layer: The query engine uses DataFusion, an open-source SQL engine written in Rust that operates natively on Arrow arrays. Instead of deserializing rows, DataFusion scans Parquet metadata (column statistics, bloom filters) to prune files, then memory-maps or reads the Parquet columns as Arrow arrays. Vectorized execution processes batches of values in registers, leveraging SIMD.

Compaction: A background Compactor periodically merges small Parquet files into larger ones, similar to LSM compaction. This reduces the number of files the query engine must scan, trading CPU (merge cost) for I/O reduction.

Horizontal Scaling: Because the engine is stateless and all data lives in cloud storage, InfluxDB IOx scales query concurrency simply by adding query nodes. The storage layer (Parquet in cloud) is unlimited; the compute layer is ephemeral. This is the lakehouse model: decouple storage and compute.

Indexing: InfluxDB IOx maintains an in-memory inverted index (tag_value → series_IDs), built at query time from Parquet metadata. This index is never persisted—it’s reconstructed on every query node. This trades memory for simplicity (no index maintenance overhead).

Advantages:
– Cloud-native: store data in S3, query from anywhere.
– Unlimited cardinality scaling: no in-memory index blowup on cardinality.
– SQL compatibility: DataFusion supports standard SQL, making it familiar to DW users.
– Open ecosystem: Apache Arrow and Parquet are industry standards.

Disadvantages:
– Latency: reconstructing indexes on every query adds microseconds; not optimized for single-digit millisecond SLA.
– Write latency: flushing to cloud storage has network overhead.
– Cost: compute nodes are always running; no true “pay-per-query” model like BigQuery.

TimescaleDB Internals

TimescaleDB is a Postgres extension that layers time-series optimizations on top of Postgres’s rock-solid transaction engine, replication, and ecosystem.

Hypertables: A hypertable is a logical table that Postgres automatically partitions by time. Create a hypertable metrics(time, tags, value), and TimescaleDB silently creates underlying chunk tables _hyper_1_1_chunk (time range T0–T1), _hyper_1_2_chunk (T1–T2), etc. To the application, metrics looks like a normal table; SELECTs and INSERTs work without modification. Behind the scenes, INSERT goes to the appropriate chunk, and SELECT prunes chunks outside the query time window.

Chunks: Each chunk is a standard Postgres heap table, subject to Postgres’s MVCC (row versioning) and locking. Chunks are immutable once full (new data goes to the next chunk). This enables aggressive optimization: indexes on full chunks are never updated, bloat never accumulates.

Continuous Aggregates: A continuous aggregate is a materialized view that is updated incrementally. Instead of:

SELECT time_bucket('1 hour', time), region, avg(value)
FROM metrics
WHERE time > now() - '30 days'::interval
GROUP BY 1, 2;

(which rescans all 30 days of data), create a continuous aggregate:

CREATE MATERIALIZED VIEW hourly_avg AS
SELECT time_bucket('1 hour', time), region, avg(value)
FROM metrics
GROUP BY 1, 2;

TimescaleDB materializes this view once, then incrementally updates it as new chunks complete. Queries on the aggregate are O(1) relative to data volume—you scan only the materialized result, not the raw data.

Columnar Compression: TimescaleDB added native columnar storage for historical data. Chunks can be compressed into columnar form, reducing storage by 90%+ while keeping query latency low. The compression uses:
– Dictionary encoding (map unique values to integers).
– Delta encoding (store differences instead of absolute values).
– Run-length encoding (compress repeated values).

A chunk can be queried in columnar form transparently—no decompression needed.

Advantages:
– Postgres compatibility: migrations from Postgres are trivial (it IS Postgres).
– ACID semantics: transactions, rollback, replication all inherited from Postgres.
– Ecosystem: pgAdmin, pg_dump, replication tools, language bindings all work.
– Cost-effective: runs on commodity servers, no vendor lock-in.

Disadvantages:
– Postgres limitations: MVCC overhead on writes; not designed for multi-terabyte cardinality.
– Latency: Postgres’s row-based engine adds CPU overhead vs. vectorized systems.
– Scaling: replication and sharding are manual or require third-party tools (Citus).

QuestDB Internals

QuestDB is a from-scratch columnar database built for ultra-low latency and high cardinality. It prioritizes performance over portability.

Memory-Mapped Columnar Storage: QuestDB stores each column as a contiguous file on disk. When you query, the OS memory-maps the file—no I/O overhead, data is paged in on demand. For a column of integers, the file is a raw array of int64 values. To sum a column, QuestDB simply iterates the memory-mapped array; the OS handles paging.

This design has radical implications:
– O(1) seeks: Jump to any row in O(1) by multiplying row_id × column_width.
– Zero-copy: Data is already in memory; no serialization/deserialization.
– CPU-friendly: Contiguous memory layout = cache hits; predictable prefetching.

Out-of-Order (O3) Ingestion: Unlike most TSDBs, QuestDB accepts writes in any order. An O3 buffer holds out-of-order data in memory; when full or on timeout, it sorts and merges with the in-order main table. This is critical for IoT and edge scenarios where devices send data late (network delays, batching).

Symbol Tables: QuestDB uses “symbols”—globally-unique dictionaries for string columns (tags). Instead of storing region="us" as a 2-byte UTF-8 string, it stores a 1-byte integer pointing into a global symbol table. This slashes cardinality from millions to thousands.

SIMD Execution: QuestDB’s query engine generates CPU-specific code (AVX2, NEON) at query time. A SUM query becomes a specialized function that sums 8 int64 values per CPU cycle. This is JIT compilation, not interpretation.

Advantages:
– Latency: sub-millisecond queries on TB-scale data; dominated by network, not compute.
– Out-of-order support: handles late-arriving data without penalty.
– Cardinality: symbol tables scale to high cardinality without bloat.
– CPU efficiency: SIMD and cache-friendly design reduce power consumption.

Disadvantages:
– Ecosystem: custom SQL dialect (not standard SQL); limited language bindings.
– Operational complexity: memory-mapping requires careful OS tuning (Linux-specific).
– Immutability: designed for append-only; updates are slow (full re-write).
– Replication: lagging behind Postgres/InfluxDB in HA features.

Compression Techniques in Time-Series Data

Compression is not optional in time-series systems. At 1 million samples/sec, an uncompressed float64 timestamp + value = 16 bytes/sample × 1M/sec × 86400 sec/day = 1.4 TB/day. Compression must reduce this by 10–100x to fit in reasonable storage.

Timestamp Compression (Delta-of-Delta):

Gorilla observes that consecutive timestamps differ by small, often-constant deltas. If samples arrive every 60 seconds:
– t₀ = 1609459200 (48 bits to store)
– t₁ = 1609459260 (delta = 60)
– t₂ = 1609459320 (delta = 60)
– t₃ = 1609459376 (delta = 56)

Store only the delta-of-delta (difference of differences):
– Δ(t₁, t₀) = 60
– Δ(Δ(t₂, t₁), Δ(t₁, t₀)) = 0 → encode as 1 bit
– Δ(Δ(t₃, t₂), Δ(t₂, t₁)) = -4 → encode as variable-length integer

If the delta-of-delta is small (±10%), encode in 1 bit; if larger, use 5–8 bits. Most time-series have nearly-constant deltas (IoT sensors poll regularly), so this achieves ~0.4 bytes per timestamp.

Float Compression (XOR):

Floating-point numbers have structure. If two samples are 72.5 and 72.51, they share the same exponent and most significant bits. Gorilla XORs consecutive samples:
– v₁ = 72.5 (IEEE 754: 0x42910000)
– v₂ = 72.51 (IEEE 754: 0x42910333)
– v₁ XOR v₂ = 0x00000333 → only 2 bytes differ

Use variable-length encoding: if XOR result has leading zero bytes, omit them. If both samples’ exponents match, the XOR is small. This achieves ~1.2 bytes per float.

Variable-Byte Encoding:

Store integers with variable length: values 0–63 fit in 1 byte; 64–16383 need 2 bytes; etc. Prefix the byte with a length indicator. Gorilla uses ZigZag encoding to handle negative numbers (−1 encodes as 1, −2 as 2, etc.), which are naturally sparse in time-series (monotonically increasing timestamps have no negatives).

Dictionary and ZSTD:

After delta-of-delta and XOR, Gorilla compresses the result with dictionary compression (mapping frequent byte patterns to shorter codes) and ZSTD. ZSTD is a modern, fast entropy coder (better than gzip, faster than bzip2).

Real-World Results:

Gorilla reported achieving 1.3 bytes per sample (timestamp + float64 value) on production Facebook metrics. This is a ~12x compression ratio vs. raw 16 bytes.

Benchmarks & Feature Matrix

Feature	InfluxDB 3 (IOx)	TimescaleDB	QuestDB
Engine	Apache DataFusion + Arrow	Postgres + custom chunks	Custom columnar
Query Language	SQL (standard)	SQL (Postgres dialect)	SQL (QuestDB dialect)
Cardinality Scaling	Excellent (metadata-driven)	Good (inverted indexes)	Excellent (symbol tables)
Cloud-Native	Yes (S3/GCS/Blob storage)	No (Postgres-style HA)	Partial (ILP protocol cloud-ready)
Open Source	Yes (Apache 2.0)	Yes (Timescale license)	Yes (Apache 2.0)
Compression Ratio	10–20x	20–50x (columnar chunks)	10–15x
Write Latency	10–50 ms (cloud flush)	<1 ms (local disk)	<1 ms (local disk)
Query Latency (p99)	100 ms–1 sec	10–100 ms	<50 ms
Out-of-Order Support	Limited	No (Postgres constraints)	Excellent (O3 buffer)
Continuous Aggregates	Partial (materialized views)	Yes (native)	Partial (downsample tables)
Ecosystem	Grafana, Telegraf, ILP protocol	pgAdmin, PostGIS, Citus	Grafana, limited third-party
Replication	Built-in (horizontal scaling)	Postgres streaming replication	Built-in (raft-based)

Edge Cases & Failure Modes

Cardinality Explosion: The silent killer. A DevOps team adds a request_id tag to their metrics for debugging. Request IDs are UUIDs: 2^128 possible values. With millions of requests/minute, cardinality climbs from thousands to billions in hours. The TSDB’s inverted index becomes too large to fit in RAM, queries slow to a crawl, ingestion stalls as the indexer battles OOM errors.

Mitigation: Define cardinality budgets (e.g., “all metrics must have <1M unique tag combinations”). Use monitoring to alert when cardinality approaches limits. Shard high-cardinality dimensions across instances. Never use unbounded IDs (UUIDs, request IDs) as tag values.

Late-Arriving Data: A sensor sends data 3 minutes late due to network outage. In InfluxDB IOx, this might land in the wrong chunk (chunks are time-range-specific). In TimescaleDB, it goes to the wrong chunk and breaks immutability guarantees. In QuestDB, the O3 buffer handles it natively.

Mitigation: Define max-acceptable lateness (e.g., “data older than 1 hour is rejected”). Use buffering windows to tolerate jitter. QuestDB is the natural choice for high-latency IoT scenarios.

Downsampling Correctness: Downsampling is lossy. A naive approach: store 1-second samples for 24 hours, then compute 1-minute averages on-the-fly. Problem: if a node crashes during downsampling, you have 23 hours of 1-second samples and 1 hour of 1-minute samples. Queries must now handle mixed resolution.

Mitigation: Pre-compute downsamples (continuous aggregates) immediately after the source chunks close. Use immutable chunk boundaries to ensure downsampling is idempotent.

Schema Evolution: A team adds a new field to their metrics (e.g., error_rate alongside response_time). In a schema-on-write system (InfluxDB, QuestDB), this is transparent—just start sending the new field. In schema-on-read systems (Parquet), you need backward compatibility (optional columns) or schema versioning.

Mitigation: Use schema-on-read systems (Parquet, Arrow) that enforce forward/backward compatibility. Avoid forced schema migrations; use ALTER TABLE carefully in Postgres-based systems.

Implementation Guide: Choosing and Deploying a TSDB

When to Pick InfluxDB 3 (IOx)

Choose InfluxDB 3 if:
– You need unlimited, elastic scaling (millions of unique metrics).
– You’re building observability (metrics, logs, traces) at scale.
– You want cloud-native architecture and pay-per-query economics.
– You prefer standard SQL and open ecosystems.

Example query:

SELECT
  time_bucket('1 minute', timestamp) as time,
  region,
  avg(cpu_usage) as avg_cpu,
  max(memory_usage) as max_mem
FROM metrics
WHERE timestamp > now() - '7 days'::interval
  AND region IN ('us-east', 'eu-west')
GROUP BY 1, 2
ORDER BY 1 DESC;

When to Pick TimescaleDB

Choose TimescaleDB if:
– You’re already using Postgres (migration ease).
– You need ACID transactions and complex JOINs (hybrid OLTP + analytics).
– You want native continuous aggregates and automatic downsampling.
– You prefer on-premise or managed Postgres infrastructure.

Example: continuous aggregate:

CREATE MATERIALIZED VIEW hourly_metrics AS
SELECT
  time_bucket('1 hour', timestamp) as time,
  host,
  avg(cpu_usage) as avg_cpu,
  stddev(memory_usage) as stddev_mem
FROM metrics
GROUP BY 1, 2;

REFRESH MATERIALIZED VIEW CONCURRENTLY hourly_metrics;

Then query the pre-computed result:

SELECT * FROM hourly_metrics WHERE time > now() - '30 days'::interval;

When to Pick QuestDB

Choose QuestDB if:
– You need sub-10 ms query latency at TB+ scale.
– You have high-cardinality data (millions of distinct tags).
– You accept late-arriving, out-of-order data.
– You’re comfortable with custom SQL dialects and limited ecosystem.

Example: symbol table and O3 ingestion:

-- Define a symbol table (global dictionary)
CREATE TABLE metrics (
  ts TIMESTAMP,
  host SYMBOL,         -- String stored as integer via symbol table
  metric_name SYMBOL,
  value DOUBLE
) TIMESTAMP(ts);

-- Ingest out-of-order data; O3 buffer sorts it
INSERT INTO metrics VALUES (
  '2026-04-18T10:00:00Z', 'host-001', 'cpu_usage', 45.2
);

-- Query with symbol-based filtering
SELECT avg(value) FROM metrics 
WHERE host = 'host-001' AND metric_name = 'cpu_usage';

Deployment Checklist

Estimate cardinality: Run a pilot, count unique (metric_name, tag_value) combinations. If >10M, use cardinality pruning or multiple instances.
Define retention: Raw 1-sec samples: 7 days? 1-min rollups: 90 days? Annual summaries: 10 years? Budget storage accordingly.
Set downsampling policy: Define rules (e.g., “after 30 days, aggregate to 1-min; after 1 year, aggregate to 1-hour”). Use continuous aggregates to automate.
Capacity planning:
– InfluxDB IOx: 1M samples/sec = ~1-2 TiB/month after compression.
– TimescaleDB: Dedicated hardware (32 GB RAM, fast SSD) handles 100k–500k samples/sec.
– QuestDB: 1–10M samples/sec on modern CPU, memory-mapped I/O.
HA setup: InfluxDB IOx (built-in via S3 replication), TimescaleDB (Postgres replication), QuestDB (raft-based replication).
Monitoring: Track cardinality, ingestion rate, query latency (p50, p99), storage growth, compaction lag.

FAQ

Q1: Is InfluxDB 3 really SQL?

Mostly yes. InfluxDB 3 uses Apache DataFusion, which implements ANSI SQL. However, some extensions (time-windowing functions, down-sampling) may differ from standard SQL. InfluxQL (InfluxDB’s legacy query language) is a custom dialect and is being phased out.

Q2: When should I pick TimescaleDB over QuestDB?

Pick TimescaleDB if you:
– Already use Postgres (skill set, infrastructure, extensions like PostGIS).
– Need ACID transactions and multi-table JOINs.
– Have moderate cardinality (<100M unique series).
– Prefer long-term vendor support and ecosystem maturity.

Pick QuestDB if you:
– Need sub-5 ms query latency on TB-scale data.
– Have high cardinality and accept out-of-order data.
– Can tolerate custom SQL and limited third-party integrations.
– Are building latency-critical financial or real-time analytics systems.

Q3: What about Prometheus for long-term storage?

Prometheus is a metrics scraper and in-memory TSDB, designed for 15-day retention. For long-term archival, Prometheus integrates with long-term storage backends: Thanos (federated querying over multiple Prometheus instances + S3), Cortex (multi-tenant, can back to InfluxDB or S3), or VictoriaMetrics (clustering, native long-term storage). Prometheus itself is not a backend; it’s an agent.

Q4: How does cardinality kill performance?

Every unique tag combination is a separate series. With 100 hosts, 100 regions, and 100 metrics, cardinality = 100 × 100 × 100 = 1M series. The TSDB maintains an inverted index (tag_value → series_IDs):
– Index memory: 1M series × 100 bytes/entry = 100 MB.
– Query planning: filtering region=us scans the index, returns 10k series, then reads them.

If cardinality hits 100M (unbounded IDs like UUIDs), the index bloats to 10+ GB, query planning becomes a bottleneck, and the system grinds to a halt. Fix: never use UUIDs, request IDs, or user IDs as tags. Instead, store them as fields or use parent-child relationships.

Q5: Can I use object storage (S3, GCS) with TimescaleDB or QuestDB?

InfluxDB IOx is designed for cloud object storage. TimescaleDB and QuestDB assume local fast storage (NVMe SSD). You can back them with networked storage (EBS, GCS Persistent Disks), but:
– Latency increases (10+ ms per I/O vs. <1 ms local SSD).
– Throughput is capped by network bandwidth.
– Cost increases (per-GB-transferred fees).

For massive scale, object storage is essential; use InfluxDB IOx or migrate to a data lakehouse (DuckDB on Parquet in S3).

Where TSDBs Are Heading

Apache Arrow & Parquet Everywhere: InfluxDB 3’s success proves that columnar formats (Arrow in-memory, Parquet on-disk) are the lingua franca of analytics. Expect all TSDB vendors to adopt them. This enables:
– Interoperability: query Parquet from DuckDB, Polars, or Apache Spark.
– Ecosystem convergence: TSDBs become nodes in data lakehouses, not silos.

Lakehouse Convergence: The future blurs TSDB, data warehouse, and data lake. Delta Lake, Apache Iceberg, and Hugging Face Parquet provide:
– Unified schema management and versioning.
– Time-travel (query historical snapshots).
– Incremental ingestion (UPSERT, low-latency appends).

Expect purpose-built TSDB engines to fade; instead, columnar storage becomes the standard, with specialized query engines (DataFusion, DuckDB, Trino) on top.

Out-of-Order as Native: Inspired by QuestDB, expect all major TSDBs to natively handle late-arriving and unordered data. This is non-negotiable for IoT and edge computing.

Continuous Aggregates as Standard: Downsampling is compute-intensive when done ad-hoc. Pre-computing rollups (continuous aggregates) is becoming table-stakes. Vendors will offer hierarchical aggregations (seconds → minutes → hours → days) out-of-the-box.

Edge TSDB Engines: TSDBs will shrink to edge devices (Kubernetes sidecars, IoT gateways). InfluxDB Edge, TimescaleDB Edge, and QuestDB Edge enable local buffering and compression, syncing to cloud on network availability.

References

InfluxDB IOx Blog: https://www.influxdata.com/blog/apache-arrow-parquet-iox/
InfluxData Documentation: https://docs.influxdata.com/influxdb/cloud/
TimescaleDB Hypertables: https://docs.timescale.com/use-timescale/latest/hypertables/
Timescale Continuous Aggregates: https://docs.timescale.com/use-timescale/latest/continuous-aggregates/
QuestDB Documentation: https://questdb.io/docs/
QuestDB Out-of-Order Ingestion: https://questdb.io/docs/guides/out-of-order/
Gorilla: A Fast, Scalable, In-Memory Time Series Database: https://www.vldb.org/pvldb/vol8/p1816-teller.pdf (Pelkonen et al., VLDB 2015)
Apache Arrow: https://arrow.apache.org/
Apache Parquet: https://parquet.apache.org/
DataFusion: https://datafusion.apache.org/
LSM Trees: https://en.wikipedia.org/wiki/Log-structured_merge-tree
Prometheus TSDB: https://prometheus.io/docs/prometheus/latest/storage/

Last Updated: April 18, 2026

Tags: time-series database, InfluxDB, TimescaleDB, QuestDB, Gorilla compression, cardinality, columnar storage, LSM trees, continuous aggregates, Apache Arrow, Parquet, observability

Time-Series Database Internals: InfluxDB 3, TimescaleDB, and QuestDB Under the Hood

Lede

TL;DR

Table of Contents

Key Concepts

What Makes a Time-Series Database Different

InfluxDB 3 (IOx) Internals

TimescaleDB Internals

QuestDB Internals

Compression Techniques in Time-Series Data

Benchmarks & Feature Matrix

Edge Cases & Failure Modes

Implementation Guide: Choosing and Deploying a TSDB

When to Pick InfluxDB 3 (IOx)

When to Pick TimescaleDB

When to Pick QuestDB

Deployment Checklist

FAQ

Where TSDBs Are Heading

References

Related

Comments

Leave a Reply Cancel reply

Tag Cloud

Categories

Lede

TL;DR

Table of Contents

Key Concepts

What Makes a Time-Series Database Different

InfluxDB 3 (IOx) Internals

TimescaleDB Internals

QuestDB Internals

Compression Techniques in Time-Series Data

Benchmarks & Feature Matrix

Edge Cases & Failure Modes

Implementation Guide: Choosing and Deploying a TSDB

When to Pick InfluxDB 3 (IOx)

When to Pick TimescaleDB

When to Pick QuestDB

Deployment Checklist

FAQ

Where TSDBs Are Heading

References

Related Posts

Related

Comments

Leave a Reply Cancel reply