DuckDB v2.0 vs 1.5.x: Benchmarks for IIoT Telemetry

DuckDB has quietly become the default query engine bolted onto edge gateways, historian backends, and lakehouse notebooks across industrial IoT stacks — it is small enough to embed in a gateway process and fast enough to replace a Spark cluster for single-node analytics. On August 17, 2026, the DuckDB team previewed the DuckDB v2.0 benchmark results for their next major release, codenamed Cyanoptera, and the numbers are not incremental: a recursive graph reachability query that took the 1.x engine 4.90 seconds now finishes in 0.12 seconds. This is not a rehash of that announcement post. We are going deeper on what actually changed under the hood — a from-scratch PEG parser, async I/O for remote object storage, and a new VARIANT type built for JSON sensor payloads — and translating both DuckDB’s own official figures and MotherDuck’s independent third-party reproduction into terms that matter for anyone querying partitioned Parquet or Iceberg telemetry from an edge gateway or a Trino-backed lakehouse.

What this covers:

• Why the parser was rewritten from a PostgreSQL-derived LALR(1) grammar to a packrat PEG parser, and what that unlocks for custom industrial functions
• How async I/O changes the economics of scanning partitioned Parquet and Iceberg tables over S3-compatible storage
• The VARIANT type as a native home for Sparkplug B and OPC UA JSON metrics, with real shredding and query-speed numbers
• A full benchmark table separating DuckDB’s own official Cyanoptera figures from MotherDuck’s independent alpha-build reproduction
• Where the alpha status of v2.0 should make you cautious before you put it in a production IIoT pipeline
• The AWS acquisition of DuckLabs, and what it means for governance risk in a production deployment

Context and Background

DuckDB’s appeal for industrial telemetry has always been structural rather than incidental. It runs in-process, needs no cluster, and reads Parquet and Iceberg directly off object storage or a local disk — which matches how most IIoT lakehouses are actually built today. Our earlier comparison of lakehouse table formats, Iceberg vs. Delta vs. Hudi: A Lakehouse ADR for 2026, covers the storage-layer decision that most DuckDB deployments sit on top of; this post picks up where that one leaves off, at the query engine itself.

The pattern shows up in three places repeatedly across industrial deployments: embedded inside an edge gateway process to answer local aggregation queries without shipping raw data upstream, as the query layer behind a historian’s ad-hoc SQL console, and as the notebook-side engine analysts reach for instead of spinning up a Spark job to profile a day’s worth of partitioned Parquet. In all three, the workload is dominated by two things — scanning remote or local Parquet files, and walking hierarchical asset or time relationships recursively — which is exactly where DuckDB v2.0’s two biggest architectural changes land.

DuckDB 1.5.5, announced around July 22, 2026, was the last point release in the 1.x line and is the baseline every benchmark in this piece measures against. On August 17, 2026, the official DuckDB blog published “A Preview of DuckDB v2.0”, describing Cyanoptera as more than a version bump: a new PEG-based SQL parser, a rewritten Storage Format v2.0 with lazy column metadata and DICT_FSST compression enabled by default, async I/O for remote storage, a VARIANT type with automatic shredding, full trigger support, NEAREST joins, DML inside CTEs, a stable and versioned Extension C API, and — notably for anyone running custom builds — signed custom extension repositories. The post states more than 10,000 commits landed since the 1.5 branch point in March 2026, which is a useful signal of how substantial this release actually is relative to a typical DuckDB point release.

Two follow-up posts matter just as much as the preview itself. On August 20, 2026, the team published a deep technical explainer on the parser rewrite, and on August 26, 2026, DuckDB and its commercial backer DuckLabs announced that DuckLabs was joining AWS while DuckDB and DuckLake remain open source. Both threads run through the rest of this piece, because both change how you should evaluate DuckDB for a production IIoT pipeline in late 2026, independent of the raw performance numbers.

Why the Engine Rewrite Matters for IIoT Telemetry Pipelines

DuckDB v2.0’s headline gains — a packrat PEG parser, async I/O, and native VARIANT storage — solve three problems that show up constantly in industrial telemetry workloads: brittle SQL generation from templated dashboards, slow scans of partitioned Parquet and Iceberg tables sitting in S3-compatible object storage, and awkward handling of semi-structured JSON payloads coming off MQTT brokers and OPC UA servers.

From LALR(1) YACC to PEG: What Actually Changed

DuckDB’s original SQL parser was derived from PostgreSQL’s grammar, built with a Bison/YACC LALR(1) parser generator. That lineage gave DuckDB broad SQL compatibility for free early on, but LALR(1) grammars have a well-known cost: every new syntax addition risks a grammar conflict, because the parser generator resolves ambiguity at build time by picking one interpretation and silently discarding the other. The August 20, 2026 post, “DuckDB v2.0: Your Database Deserves a Better Parser”, frames this plainly: extending the grammar had become the single biggest source of friction in adding new syntax to DuckDB, because conflicts are often invisible until a specific query triggers the wrong branch.

The replacement is a packrat PEG (parsing expression grammar) parser, which resolves ambiguity by ordered choice instead of table-driven conflict resolution, and memoizes sub-parses so it doesn’t re-derive the same expression tree repeatedly on backtracking. The DuckDB team’s own stress case makes the performance delta concrete: a malformed query with 19 unmatched parentheses — the kind of thing a buggy code generator or a fuzzer produces — took 10.6 seconds to fail on the old LALR(1) parser and 0.001 seconds on the new packrat parser. That’s not a typical query, but it is exactly the kind of degenerate input that shows up when SQL is generated programmatically, which describes most IIoT query layers: historian adapters, Grafana panels backed by templated SQL, and BI tools that assemble WHERE clauses from user-selected asset filters.

What Runtime Grammar Extension Unlocks for Custom Industrial Functions

The practical unlock for industrial users is runtime grammar extension. Because the new parser can extend its own grammar at runtime rather than requiring a recompiled build, DuckDB extensions can register new SQL syntax — not just new functions — without a custom fork. For teams building domain-specific functions around Sparkplug B metric names, OPC UA node IDs, or asset-hierarchy path syntax, that’s the difference between bolting a UDF onto standard SQL and actually extending the language DuckDB speaks.

Concretely, this is the difference between writing SELECT sparkplug_metric(payload, 'Temperature/Value') as a function call and being able to register a first-class syntax extension such as an asset-path operator that walks a hierarchy without nested function calls, or a custom PIVOT-like clause tuned to how Sparkplug B groups metrics under a device birth certificate. The preview post is explicit that new capabilities also include expression statements — queries that skip the SELECT keyword entirely for quick expression evaluation — and new CONNECT and DISCONNECT statements for the Quack server protocol, plus COPY TO with PARTITION BY and ORDER BY clauses, which matters directly for teams writing partitioned telemetry Parquet output straight from a query. The post also states no breaking changes are expected for typical queries — same DuckSQL syntax, same semantics, just a different execution path underneath.

Async I/O Meets Partitioned Parquet and Iceberg

The direct answer: DuckDB v2.0’s async I/O engine overlaps network waits with CPU work when scanning remote Parquet and Iceberg files, instead of blocking a thread per request; DuckDB’s own preview cites broad gains, and MotherDuck’s independent alpha reproduction measured roughly 2.1-3x faster S3 Parquet and CSV scans against a 1.5.5 baseline — the exact access pattern IIoT lakehouse queries depend on.

Most industrial telemetry lakes are not a single tidy file. They’re partitioned Parquet — by site, by asset, by day — sitting behind an Iceberg or DuckLake catalog on S3-compatible storage, which is the pattern our Trino vs. Presto vs. Apache Spark: Lakehouse Query Engines for 2026 comparison assumes throughout. Scanning that layout means issuing many range requests against object storage, and the old synchronous I/O model in DuckDB 1.x blocked worker threads on each request, serializing network latency into wall-clock query time whenever a query touched more files than there were I/O threads to service them.

Async I/O changes that by decoupling request issuance from thread occupancy — DuckDB can have many storage requests in flight while CPU-bound decompression and filtering proceed on completed chunks, rather than waiting on each file sequentially. This is precisely the class of workload IIoT telemetry queries stress hardest: a dashboard panel asking for the last 30 days of vibration readings from 40 machines is, physically, dozens of partition reads against S3, not one. There’s a secondary, less obvious benefit here too: faster scans mean shorter-lived compute for the same query, which on metered cloud query engines and serverless DuckDB deployments translates directly into lower per-query compute cost, not just lower latency.

VARIANT: A Native Home for Sparkplug B and OPC UA JSON

Industrial telemetry payloads are rarely flat. Sparkplug B messages carry nested metric arrays, OPC UA structured values carry variant-typed fields, and most historian adapters land these as JSON strings because forcing them into a rigid relational schema ahead of time is brittle against firmware and metric-set changes. DuckDB 1.x could query JSON, but only by parsing the string on every access — no persistent structure, no columnar pruning, no compression tuned to the shape of the data.

VARIANT changes that by shredding semi-structured values into an internal columnar representation automatically, while still accepting arbitrary JSON on ingest and integrating directly with Parquet’s own logical type system, so a VARIANT column round-trips through Parquet without a lossy string-encoding detour. That matters specifically for MQTT and Sparkplug B ingestion pipelines, where the JSON structure is often nested inconsistently across firmware versions and forcing a rigid schema at write time breaks the pipeline every time a device vendor changes its metric set.

Figure 1: DuckDB v2.0’s Cyanoptera architecture — a packrat PEG parser feeds the query planner, which routes to the async I/O layer for remote storage, the new Storage Format v2 with DICT_FSST compression, the VARIANT engine for semi-structured shredding, the Quack server protocol, and the new trigger engine.

Long description: The diagram shows SQL query text entering a PEG packrat parser, which passes a parsed plan to a central query planner. From the planner, four branches fan out: one to the async I/O layer serving remote S3-style storage, one to Storage Format v2 with lazy column metadata feeding DICT_FSST compression, one to the VARIANT type engine performing automatic shredding, and one to the new Quack server protocol handling CONNECT and DISCONNECT, alongside a trigger engine supporting transition tables.

Figure 4: How a Sparkplug B payload moves from an MQTT broker through DuckDB’s VARIANT ingest path into Parquet storage, ending in direct SQL field access without a JSON re-parse.

Long description: A Sparkplug B payload arrives at an MQTT broker, is picked up by a DuckDB ingest function, stored as a VARIANT column, automatically shredded, written to a Parquet file carrying VARIANT metadata, and finally queried with SQL that accesses a specific metric value field directly rather than re-parsing the raw JSON text on every read.

Benchmark Walkthrough: Async I/O, Recursive CTEs, and VARIANT Field Access

Two distinct sets of numbers are in play here, and conflating them is the single easiest way to misquote this release. DuckDB’s own preview post cites its recursive-CTE, timezone-conversion, and collation-filter figures directly. MotherDuck published a separate hands-on reproduction on September 10, 2026, benchmarking the public v2.0.0-alpha39998 build against 1.5.5 on S3 Parquet scans, CSV reads, a 20,000-commit git-ancestry recursive query, and VARIANT storage and query performance. The git-ancestry recursive CTE result is MotherDuck’s own separate test — not a restatement of DuckDB’s official 40x figure — and the table below keeps every row attributed to its actual source.

Figure 2: How the three headline benchmark categories — async S3 Parquet scanning, recursive CTE reachability, and VARIANT field access — break down between DuckDB 1.5.5 and the v2.0 alpha build.

Long description: A benchmark suite branches into three test categories. The async S3 Parquet scan branch shows 18.8 seconds on 1.5.5 versus 7.7 seconds on v2.0 alpha. The recursive CTE reachability branch shows 4.90 seconds on 1.5.5 versus 0.12 seconds on v2.0. The VARIANT field access branch shows a 1.5.5 VARIANT baseline versus roughly 78 times faster field-level queries on v2.0.

Figure 3: A typical IIoT telemetry query path — field sensors over OPC UA and MQTT into an edge gateway running embedded DuckDB v2.0, writing local Parquet, uploading asynchronously to S3-compatible storage, and surfacing through an Iceberg table layer to dashboards.

Long description: Field sensors using OPC UA and MQTT protocols feed an edge gateway. The gateway runs embedded DuckDB v2.0, which writes local Parquet files. An async I/O upload step moves those files to S3-compatible storage, which is organized by an Iceberg table layer, which in turn feeds a historian query layer and finally dashboards and alerts.

The 40-60 word answer: async I/O and the PEG parser combine to make DuckDB v2.0 meaningfully faster at exactly the operations IIoT pipelines run constantly — scanning partitioned remote Parquet, walking recursive asset hierarchies, and reading VARIANT-shredded JSON telemetry — with DuckDB’s official recursive-CTE figure and MotherDuck’s independent alpha reproduction agreeing directionally even though they measured different queries.

Benchmark	Metric	DuckDB 1.5.5	DuckDB v2.0 (alpha/preview)	Change	Source
Recursive CTE reachability query	Query time	4.90 s	0.12 s	~40x faster	DuckDB official, Aug 17 2026 preview
Timezone conversion, 25M rows	Query time	baseline	—	2.2x faster	DuckDB official, Aug 17 2026 preview
German-collation filter, 5M rows	Query time	baseline	—	2.6x faster	DuckDB official, Aug 17 2026 preview
Malformed SQL, 19 unmatched parens	Parse time	10.6 s	0.001 s	~10,000x faster	DuckDB official, Aug 20 2026 parser post
Single 2.2GB Parquet file over S3	Scan time	18.8 s	7.7 s	~2.4x faster	MotherDuck third-party, Sep 10 2026, alpha build
23-file, 13.6GB Parquet set over S3	Scan time	11.8 s	3.9 s	~3x faster	MotherDuck third-party, Sep 10 2026, alpha build
1.7GB CSV read	Read time	116 s	55 s	~2.1x faster	MotherDuck third-party, Sep 10 2026, alpha build
Recursive CTE, 20,000-commit git ancestry	Query time	1.8-16 s (depth-dependent)	~0.10 s (flat)	up to ~160x faster	MotherDuck third-party, separate reproduction
VARIANT storage, 224MB raw JSON	Storage size	81MB (1.5.5 VARIANT)	85MB (v2.0 VARIANT)	roughly comparable	MotherDuck third-party, Sep 10 2026, alpha build
VARIANT field-level query	Query time	baseline	~78x faster than 1.5.5 VARIANT	~78x faster	MotherDuck third-party, Sep 10 2026, alpha build

A methodology note that should not be skipped: every MotherDuck figure in that table was measured against a public alpha build — tagged v2.0.0-alpha39998 on the v2.0-cyanoptera branch, announced September 2, 2026 with a feature freeze in effect — not the stable GA release. DuckDB’s own official numbers come from a separate set of internal benchmarks published in the August 17 and August 20 preview posts, run on their own pre-release builds. Neither source represents final GA performance, which DuckDB currently projects for the second half of October 2026. Treat every number here as directionally reliable evidence of where the architecture is headed, not as a guarantee of what a shipped v2.0.0 release will measure on your own hardware, your own S3 provider’s latency profile, or your own file layout.

It’s also worth flagging that the recursive-CTE numbers in the table come from two different queries entirely — DuckDB’s own graph reachability benchmark, and MotherDuck’s git-commit-ancestry benchmark — and both happen to land in a similar multiples-of-tens-to-hundreds speedup range. That agreement across independent test cases is a stronger signal than either number alone, but it’s still two separate experiments, not one figure repeated by two sources. Neither benchmark, notably, isolates how much of the gain comes from the PEG parser versus the underlying execution engine changes — for a query this simple, parse time is a negligible fraction of total runtime, so the bulk of a 40x gain on a recursive reachability query is almost certainly execution-engine work rather than parsing, even though this piece has spent considerable time on the parser story for its own architectural merits.

Trade-offs, Gotchas, and What Goes Wrong

The most important caveat is the one the benchmarks themselves already flag: every third-party number here comes from a pre-GA alpha build. DuckDB has declared a feature freeze on v2.0.0-alpha39998, which is a meaningfully more stable signal than an early preview, but “feature freeze” is not “release candidate,” and the team’s own guidance still projects stable GA for the second half of October 2026. Running an alpha build against production IIoT telemetry — data that often feeds compliance reporting or safety-adjacent dashboards — carries real risk that has nothing to do with the performance numbers being wrong; it’s about undiscovered edge cases in unreleased code paths.

The PEG parser rewrite is described as accepting the same DuckSQL syntax with no breaking changes expected for typical queries, but “typical” is doing real work in that sentence. Any pipeline that depends on a specific parser error message, a specific ambiguity resolution the old LALR(1) grammar happened to pick, or an undocumented syntax quirk should be re-tested against the new parser before cutover, not assumed compatible by default. Query-generation layers that build SQL programmatically — exactly the historian and dashboard adapters this article is written for — are the most likely place a subtle parsing difference would surface, because they’re the layers most likely to produce the kind of edge-case input the old grammar silently favored one way and the new parser might resolve differently.

The VARIANT storage numbers deserve a second look too. MotherDuck’s own reproduction found VARIANT storage on v2.0 (85MB) is actually slightly larger than VARIANT storage on 1.5.5 (81MB) for the same 224MB raw JSON input — the win is in field-level query speed, not storage footprint. If your evaluation criteria are storage cost first and query speed second, that trade-off runs the opposite direction you might assume from a release billed around performance gains.

The removal of ICU in favor of native timezone and calendar handling is another migration item worth a dedicated test pass, even though it’s framed as a maintenance simplification rather than a feature cut. Any pipeline doing timezone-sensitive aggregation across multi-site telemetry — the exact scenario the 2.2x timezone-conversion benchmark targets — should validate that locale and calendar edge cases (leap seconds around daylight-saving transitions, non-Gregorian calendar handling if used anywhere) behave identically under the native implementation before relying on it for cross-site rollups.

Signed custom extension repositories are a governance improvement, but they also mean any internally maintained DuckDB extension — a custom Sparkplug B parser, say — will eventually need to go through a signing process to keep working smoothly against a hardened v2.0 install. Teams running unsigned internal extensions today should treat this as a forward-looking action item, not an immediate blocker, since the alpha still confirms core extensions like httpfs, ducklake, iceberg, and spatial working without issue.

Practical Recommendations

Given the alpha status of v2.0 and the strength of the underlying gains, the right posture for most IIoT teams in September 2026 is evaluate now, deploy after GA. The engineering story — async I/O, a PEG parser built for extensibility, and native VARIANT support — is compelling enough to justify real testing time against your own telemetry schemas, but not compelling enough to justify running unreleased database code against production safety or compliance data.

Concretely:

• Install the alpha in a non-production environment via curl https://install.duckdb.org | DUCKDB_VERSION=alpha bash or pip install duckdb --pre --upgrade, and confirm your core extensions — httpfs, ducklake, iceberg, spatial — are the ones you actually depend on; DuckDB has confirmed these work in the alpha.
• Re-run your own recursive-CTE and Parquet/Iceberg scan queries against the alpha build with your real partition layout and your real object storage provider, rather than trusting either DuckDB’s or MotherDuck’s numbers to transfer directly to your environment.
• If you ingest Sparkplug B or OPC UA JSON today via string columns and manual parsing, prototype a VARIANT-backed ingest path now — the ~78x field-query speedup and Parquet-native shredding are the most operationally relevant wins in this release for telemetry-heavy workloads.
• Stress-test any programmatic SQL generation layer — historian adapters, templated dashboard queries — against the new PEG parser specifically, since that’s where an undocumented old-parser quirk is most likely to surface as a behavioral difference.
• Validate timezone-sensitive multi-site aggregation queries against the native (post-ICU) timezone handling before relying on it for cross-site rollups.
• Track the AWS/DuckLabs relationship as a governance input to your vendor risk process, separate from your technical evaluation of the engine itself.
• Set a calendar reminder for the projected GA window (second half of October 2026) rather than tracking alpha builds indefinitely; re-benchmark against the actual GA release before any production migration decision.

Frequently Asked Questions

Is DuckDB v2.0 stable enough for production IIoT pipelines right now?

Not yet, as of late September 2026. The current build is a public alpha — v2.0.0-alpha39998 on the v2.0-cyanoptera branch — with a feature freeze in effect but no release candidate. DuckDB projects stable GA for the second half of October 2026. Evaluate it now in staging against your own telemetry queries, but hold production cutover until a GA release, especially for pipelines feeding compliance or safety-relevant dashboards.

How much faster is DuckDB v2.0 than 1.5.5 for querying Parquet on S3?

MotherDuck’s independent alpha-build reproduction measured roughly 2.4x faster on a single 2.2GB Parquet file, about 3x faster across a 23-file, 13.6GB set, and about 2.1x faster on a 1.7GB CSV read, all against a DuckDB 1.5.5 baseline. These are third-party numbers against a pre-GA build, not DuckDB’s own official figures, so treat them as directional evidence rather than guaranteed GA performance.

What is the VARIANT type and why does it matter for MQTT and Sparkplug B telemetry?

VARIANT is a new column type that stores semi-structured data — like JSON from MQTT or Sparkplug B payloads — with automatic shredding into an internal columnar layout and native Parquet integration, instead of storing it as a string that gets re-parsed on every query. MotherDuck’s benchmark found VARIANT field-level queries roughly 78x faster than 1.5.5’s VARIANT implementation and about 6x faster than parsing equivalent JSON text on v2.0 itself.

Why did DuckDB rewrite its SQL parser from LALR(1) to PEG?

The original parser, derived from PostgreSQL’s Bison/YACC LALR(1) grammar, made extending SQL syntax risky because new grammar rules could silently conflict with existing ones. The new packrat PEG parser resolves ambiguity by ordered choice and memoizes sub-parses, avoiding that conflict class entirely. It also enables runtime grammar extension, letting extensions register new syntax rather than just new functions — relevant for teams building custom industrial-data query functions.

Does the AWS acquisition of DuckLabs affect whether I should use DuckDB?

DuckLabs, the commercial entity behind DuckDB, was acquired by AWS as announced on August 26, 2026, but DuckDB and DuckLake are explicitly stated to remain open source. This is a governance signal worth tracking in a vendor risk process — particularly for teams already invested in AWS-native lakehouse tooling — rather than a reason to avoid or delay adoption of the open-source engine itself.

Should I merge DuckDB’s official benchmark numbers with MotherDuck’s third-party numbers into one performance claim?

No. DuckDB’s official figures — the 40x recursive-CTE reachability result, the 2.2x timezone-conversion gain, and the 2.6x collation-filter gain — come from the company’s own preview posts and internal test builds. MotherDuck’s numbers are an independent hands-on reproduction against the public alpha, including a separate git-ancestry recursive-CTE test. Both point in the same direction, but they measured different queries on different builds, and should be cited with their separate attributions intact.

Will the Quack server protocol change how DuckDB fits into an edge gateway architecture?

Potentially, though it’s early. The Quack server protocol adds CONNECT and DISCONNECT statements that let DuckDB behave more like a server process rather than purely an embedded library, which could simplify architectures where multiple edge processes need to share one DuckDB instance instead of each embedding their own copy. This is a newer, less-benchmarked feature relative to async I/O and VARIANT, so validate it carefully against your own multi-process gateway design before depending on it.

Further Reading

• Iceberg vs. Delta vs. Hudi: A Lakehouse ADR for 2026
• Trino vs. Presto vs. Apache Spark: Lakehouse Query Engines for 2026
• InfluxDB vs. TimescaleDB vs. ClickHouse: IoT Time Series in 2026
• DuckDB, “A Preview of DuckDB v2.0”, official DuckDB blog, August 17, 2026
• MotherDuck, “Why DuckDB 2.0 is faster”, September 10, 2026

By Riju — about