Change Data Capture with Debezium: Streaming Architecture (2026)

Every dual write is a lie waiting to be discovered. You update a row in Postgres, then publish an event to Kafka — and somewhere between those two calls the process crashes, the network partitions, or a retry storm fires the event twice. Your database and your event stream now disagree, and no amount of downstream reconciliation fully fixes it. Change data capture debezium pipelines exist to eliminate that class of bug entirely: instead of asking your application to write twice, you read the database’s own transaction log — the single, ordered, already-committed record of truth — and turn every committed change into a stream event. The log was going to be written anyway. Debezium just reads it.

This post is an architecture decision record, not a quickstart. It explains why log-based CDC beats query-based and trigger-based approaches, exactly how Debezium reads Postgres WAL and MySQL binlog, how the Kafka Connect topology is wired, and where these pipelines fail in production.

What this covers: the three CDC strategies and why log-based wins; the transaction-log reader mechanics; Kafka Connect topology with converters, SMTs, and dead-letter queues; snapshots and signaling; schema evolution; delivery semantics; the outbox pattern; and the failure modes that page you at 3 a.m.

Context and Background

Change data capture is the practice of identifying and propagating row-level changes from a source database so other systems can react to them. The problem is old — data warehouses have needed incremental loads for decades — but the requirements have hardened. Modern systems want changes in seconds, not in a nightly batch; they want every intermediate state, not just the final one; and they want the propagation to be reliable enough to drive cache invalidation, search indexing, audit trails, and microservice choreography.

The incumbent approaches were query-based (poll a table on an updated_at column) and trigger-based (fire a database trigger that writes to a shadow table). Both work at small scale and both break at large scale, for reasons we will make precise below. The shift that changed the industry was log-based CDC, popularised by Debezium, an open-source project built on Kafka Connect. Debezium reads the write-ahead log that the database already maintains for crash recovery and replication, so it captures every committed change in commit order with near-zero added load on the primary.

The broader context is the move toward event-driven and streaming data architectures. Once changes are on Kafka, they fan out to warehouses, lakehouse table formats, search clusters, caches, and other services. CDC has become the connective tissue of the modern data platform — the thing that keeps a lakehouse table format in sync with the operational database that feeds it. If you are designing that layer in 2026, log-based CDC with Debezium is the default, and the decision you actually have to make is how to operate it safely.

The Log-Based CDC Reference Architecture

Log-based CDC works because relational databases already keep an ordered, durable record of every committed change for their own crash recovery and physical replication. Debezium attaches to that record as if it were a replica, decodes each change into a structured event, and hands it to Kafka Connect for delivery to Kafka. The application is never involved and never modified.

Figure 1: The log-based CDC reference architecture. Application writes hit the source database, which records them in its transaction log. Debezium reads the log, Kafka Connect serialises events through a schema registry into per-table topics, bad records divert to a dead-letter queue, and multiple sinks consume independently.

The flow in Figure 1 runs left to right. Application writes commit to the source database. The database appends each commit to its transaction log — the WAL in Postgres, the binlog in MySQL. Debezium’s source connector, running inside the Kafka Connect runtime, streams that log, decodes each change, and produces one event per row change. A converter serialises the event, optionally consulting a schema registry, and Connect writes it to a Kafka topic named after the table. Records that cannot be processed divert to a dead-letter queue. Downstream, a warehouse loader, a search indexer, and a cache invalidator each consume the same topics at their own pace.

Why the transaction log is the right source

The transaction log is the database’s source of truth for durability. Every committed change is there, in commit order, before the change is even acknowledged to the client. Reading it gives you three properties no other method can match together: completeness (you see every change, including ones that are immediately overwritten), ordering (changes arrive in the exact sequence they committed), and low overhead (you are reading a sequential file the database already writes, not running extra queries against live tables).

Crucially, log-based CDC captures deletes. A query-based poller that selects rows with a recent updated_at never sees a DELETE, because the row is gone. Trigger-based CDC can capture deletes but adds synchronous write amplification to every transaction. The log has already recorded the delete as a first-class event, so Debezium emits it naturally with the row’s prior state in the before field.

The event envelope carries before, after, and source

A Debezium change event is not a bare row. It is an envelope with an op field (c create, u update, d delete, r snapshot read), a before image, an after image, and a source block with the log coordinate, transaction ID, table name, and timestamp. This structure is what makes downstream consumers powerful: an audit system reads before and after to reconstruct exactly what changed; a cache invalidator reads the primary key from after or before; a warehouse loader upserts on after for creates and updates and deletes on before for deletes. The source block lets consumers deduplicate and reason about ordering.

Topics are partitioned by primary key for per-key ordering

Debezium routes each table to its own topic and, by default, uses the row’s primary key as the Kafka message key. Because Kafka guarantees ordering within a partition and the key determines the partition, all changes to a given row land in the same partition in commit order. That per-key ordering is the guarantee downstream systems actually need — you do not care whether row A changed before row B, but you care intensely that row A’s three updates arrive in the right sequence. This is why a naïve “just increase partitions for throughput” change is dangerous: repartitioning without preserving key affinity can reorder a row’s history.

How Debezium Reads the Transaction Log

The interesting engineering is in how Debezium attaches to each database’s log. The Postgres and MySQL paths differ enough that operating them safely requires understanding both. This section is the walk-through: the mechanics, the configuration that matters, and the sizing math that tells you when a slot will bloat your disk.

Figure 2: Snapshot then stream. Debezium first reads existing rows as r events, records the log position where streaming should begin, emits a completion marker, then switches to tailing the log and emitting c, u, and d events. Sinks upsert by primary key so replay is idempotent.

PostgreSQL: logical replication slots and pgoutput

On Postgres, Debezium acts as a logical replication client. Logical replication decodes the physical WAL into a logical stream of row changes using an output plugin. Debezium supports the native pgoutput plugin, which ships with Postgres and requires no extra extension. To enable it you set wal_level = logical, provision max_replication_slots and max_wal_senders, and grant the connector’s role replication privileges.

The critical object is the replication slot. A slot is a named, server-side cursor that records the log sequence number (LSN) up to which the consumer has confirmed receipt. Its job is to guarantee that Postgres will not recycle WAL segments the consumer has not yet read — even if the consumer is offline. That guarantee is exactly what makes CDC durable across connector restarts, and it is also the single most dangerous thing about running Debezium on Postgres.

Here is the failure mode in numbers. Suppose your database generates 40 MB of WAL per minute under normal load, and your Debezium connector dies at 02:00 during a deploy. The slot pins the WAL at the connector’s last confirmed LSN. Every minute the connector stays down, Postgres retains another 40 MB it cannot recycle: 2.4 GB per hour, roughly 57 GB per day. If your data volume is provisioned with 100 GB of free headroom, you have under two days before pg_wal fills the disk — and a full pg_wal volume takes the entire primary down, not just CDC. This is why you monitor pg_replication_slots for restart_lsn lag and set max_slot_wal_keep_size as a safety valve so Postgres will invalidate a runaway slot rather than crash the database. Invalidation loses the CDC position and forces a fresh snapshot, but a fresh snapshot beats an outage.

MySQL: the binary log and GTIDs

On MySQL, Debezium reads the binary log, the same stream MySQL replicas consume. The binlog must be enabled and set to ROW format so that full before and after row images are recorded rather than the statements that produced them. Debezium connects as a replica, requests the binlog from a recorded position, and decodes each event.

Position tracking is the difference. Classic MySQL replication tracks a binlog file name and offset, which is fragile across failovers because coordinates differ between servers. Global transaction identifiers (GTIDs) assign each transaction a cluster-unique ID, so when a primary fails over the connector can resume at the right transaction on the new primary without guessing an offset. GTIDs are not strictly required, but for any production topology with replicas and failover you want them on. MySQL’s analog to the slot-bloat problem is binlog retention: binlog_expire_logs_seconds controls how long binlogs are kept, and if the connector lags past that window the binlog it needs is purged and it must re-snapshot.

Schema history: the connector must know the past schema

Because a change event needs the column layout that was in effect when the change committed, Debezium maintains its own schema history. On MySQL the connector reads DDL statements from the binlog and persists them to a dedicated schema history Kafka topic; on restart it replays that topic to rebuild the table structures before it resumes streaming. This is why that internal topic must be a single-partition, highly durable, infinitely retained topic — losing it means the connector cannot interpret older log positions and needs a new snapshot. It is a small topic that is disproportionately important to protect.

Snapshots, Signaling, and Kafka Connect Topology

A CDC connector cannot start by streaming the log alone, because the log does not contain the rows that already existed before the connector was created. It has to reconcile the current state of every captured table with the ongoing change stream. How it does that, and how the Kafka Connect runtime is wired around it, is the operational heart of the system.

Initial, incremental, and ad-hoc snapshots

When a Debezium connector first starts, it performs an initial snapshot: it reads the full contents of each captured table and emits each row as an r (read) event, then records the log position at snapshot start and switches to streaming from there. On a large table this can take hours and, in older versions, could hold locks or block streaming.

The modern answer is the incremental snapshot, defined in Debezium’s DDD-3 design document. Instead of one giant blocking read, the connector snapshots each table in configurable chunks while streaming continues concurrently. It uses a watermarking method: it opens a snapshot window, reads a chunk, closes the window, and deduplicates any rows that changed during the window against the live stream. This means a snapshot no longer blocks change capture, and it can be paused and resumed.

The unlock is ad-hoc snapshots via signaling. You send an execute-snapshot signal — by inserting a row into a signaling table or posting to a signal topic — naming the tables to re-snapshot. This is how you backfill a newly added table, recover a sink that fell behind, or re-seed a downstream store without restarting the connector or replaying the entire log. Signaling turns snapshots from a one-time bootstrap into an operational tool you use routinely.

Kafka Connect: connectors, converters, SMTs, and the DLQ

Debezium is a plugin for the Kafka Connect framework, and Connect supplies the runtime, distribution, and fault tolerance. A Debezium Kafka Connect source connector runs as a set of tasks distributed across a cluster of worker processes. Each worker is stateless; offsets, connector configs, and task status live in internal Kafka topics, so a worker can die and its tasks rebalance onto survivors.

Three transformation stages sit between the connector and Kafka. Converters serialise the event key and value — Avro and Protobuf converters integrate with a schema registry and produce compact, schema-checked payloads, while the JSON converter is human-readable but larger. Single Message Transforms (SMTs) are lightweight per-record functions applied inline: unwrapping the Debezium envelope to a flat row, re-routing topics, masking a column, or filtering. The dead-letter queue is the safety net: when a record cannot be serialised or a transform throws, Connect can route the offending record to a configured DLQ topic instead of halting the task, so one poison record does not stop the pipeline.

One connector or many, and where SMTs run

A design choice that bites teams later is connector granularity. One connector capturing 200 tables is simple to deploy but couples the fate of all 200 topics — a schema problem on one table can stall the shared task, and a re-snapshot reloads everything. Splitting into several connectors by domain isolates blast radius at the cost of more slots (on Postgres, more slots means more WAL-retention risk) and more moving parts. The pattern that scales is a small number of connectors grouped by change-rate and criticality, each with its own slot budget, so a hot high-churn table does not share a task with cold reference data. Keep SMTs cheap — they run on the Connect worker’s hot path, and an expensive transform becomes a throughput ceiling for every record.

Delivery Semantics, Schema Evolution, and the Outbox Pattern

The two questions every architect asks about a CDC pipeline are “will I lose or duplicate events?” and “what happens when the schema changes?” The honest answers require care, because the marketing word “exactly-once” hides a lot of conditions.

At-least-once is the default; design for it

Debezium on Kafka Connect delivers at-least-once by default. If a worker crashes after producing an event to Kafka but before committing the source offset, on restart it re-reads from the last committed offset and re-emits events it already sent. You will see duplicates. You will not silently lose committed changes, which is the guarantee that matters, but you must design consumers to tolerate repeats.

The practical defence is idempotency. Because every Debezium event carries the primary key and a monotonic log coordinate in source, downstream sinks can dedupe cheaply. A warehouse or key-value sink that upserts by primary key is naturally idempotent — applying the same event twice yields the same row. A consumer that must perform non-idempotent side effects should record processed event coordinates and skip ones it has seen. This upsert-dedupe pattern is why the sinks in Figure 2 apply changes by key: it converts at-least-once transport into effectively-once state.

The caveats around exactly-once

Kafka added exactly-once support for source connectors in KIP-618 (Kafka 3.3), which lets a source connector write records and commit offsets in one transaction so records from uncommitted transactions never become visible. Debezium has moved toward supporting this, but “exactly-once” here means exactly-once production into Kafka, not exactly-once end to end. The moment an event leaves Kafka for an external sink over a non-transactional protocol, the two-generals problem returns and you are back to idempotent consumers. Treat end-to-end exactly-once as a property you engineer with idempotency and deduplication, not a checkbox the connector gives you. This is the same discipline that governs any async processing architecture: assume retries, make handlers idempotent.

CDC schema evolution and the registry

Source schemas change — a column is added, a type widens, a table is renamed. CDC schema evolution is the discipline of letting the stream absorb those changes without breaking consumers. The mechanism is a schema registry paired with a compatibility policy. When Debezium emits an event, the Avro or Protobuf converter registers the event’s schema under the topic’s subject and stamps the payload with a schema ID; consumers fetch the schema by ID to deserialise. The registry enforces a compatibility rule — typically backward compatibility, meaning new schemas can read data written under old ones.

Figure 3: Schema evolution and failure handling. Each event is serialised and checked against the registry. Backward-compatible changes register a new version; breaking changes are rejected at the source rather than corrupting the stream. At the sink, records that fail deserialisation retry and, once retries are exhausted, divert to a dead-letter queue for alerting and manual replay.

Figure 3 shows why doing this at the boundary matters. Additive, backward-compatible changes — a new nullable column — register a new schema version and flow through; consumers that do not know the column ignore it. Breaking changes — dropping a column a consumer depends on, or narrowing a type — are caught at registration and rejected, so a bad migration fails loudly at the pipeline edge instead of silently poisoning every downstream store. Records that still fail at a sink retry, and on exhaustion divert to the DLQ, where they can be inspected and replayed after a fix rather than blocking the partition.

The transactional outbox pattern

The hardest problem CDC solves is the dual write. A service that both updates its database and publishes an event has two systems to keep consistent, and no single transaction spans them. The outbox pattern cdc solution is elegant: write the event into an outbox table inside the same local transaction that changes the business data, and let Debezium turn that outbox row into a Kafka event.

Figure 4: The transactional outbox. A single database transaction writes both the business row and an outbox row, so they commit or roll back together. Debezium reads both from the transaction log and its outbox-router transform publishes the outbox entry to the correct event topic, where idempotent consumers apply it.

Figure 4 shows the guarantee. Because the business change and the outbox event are one atomic transaction, they are consistent by construction — there is no window where one exists without the other. The log records both. Debezium’s outbox event router SMT recognises outbox rows, uses their columns to set the destination topic, message key, and headers, and publishes a clean domain event rather than a raw table change. Consumers receive a purposeful event — OrderPlaced, not INSERT into orders — with the same at-least-once semantics, so they still dedupe by the event’s ID. The outbox pattern is how you get reliable event publishing from a service without distributed transactions, and it is the single most valuable pattern in this whole space. One cost worth naming: the outbox table and its churn add write and vacuum load to the source, so you delete or partition consumed rows aggressively.

Trade-offs, Gotchas, and What Goes Wrong

Log-based CDC is the right default, but it fails in specific, recurring ways. Naming them is the point of an ADR.

Replication-slot bloat and disk fill. Covered in the sizing math above, this is the top Postgres incident. An abandoned or lagging slot pins WAL until the disk fills and the primary crashes. Mitigations: monitor slot lag continuously, alert well before the disk fills, set max_slot_wal_keep_size, and always drop the slot when you decommission a connector. A forgotten slot from a deleted connector will quietly kill a database weeks later.

Connector rebalancing storms. Kafka Connect redistributes tasks when workers join or leave. During a rebalance, tasks stop and resume, which can pause capture and, on resume, re-emit a few events (at-least-once again). Frequent worker restarts — from an aggressive autoscaler or an OOM loop — cause repeated rebalances that stall throughput. Right-size worker memory, and prefer incremental cooperative rebalancing to avoid stop-the-world pauses.

Poison messages and DLQ discipline. A record that cannot be serialised or transformed will, without a DLQ, halt the task and freeze the partition. Configure the dead-letter queue, but then actually watch it — a DLQ nobody monitors is just silent data loss with extra steps. Alert on DLQ arrival and have a documented replay procedure.

Ordering assumptions. Per-key ordering holds only while the key maps stably to a partition. Changing partition count, changing the message key via an SMT, or fanning a table across topics can reorder a row’s history. Any repartitioning of a CDC topic needs explicit thought about key affinity, and consumers that assume global ordering across keys are simply wrong.

Snapshot cost and long-running transactions. An initial snapshot of a multi-terabyte table is expensive and, on Postgres, a long-running snapshot transaction can itself block vacuum and bloat the catalog. Prefer incremental snapshots, and on very large tables snapshot during low-traffic windows. Also beware very long source transactions: Debezium buffers a transaction’s changes until commit, so a transaction that stays open for hours delays every event inside it.

Toast and large values on Postgres. Postgres stores oversized column values out-of-line (TOAST), and unchanged TOASTed columns are not present in the logical decoding output. Debezium emits a placeholder for them, and a naïve consumer can mistake “unchanged, not sent” for “set to null.” Configure REPLICA IDENTITY FULL on tables where you need complete before-images, accepting the extra WAL it generates.

Practical Recommendations

Start log-based, not query-based. Unless you are on a database with no usable log access, reach for Debezium first; query-based polling is a fallback for constrained environments, not a design goal. Build the pipeline assuming at-least-once and make every consumer idempotent from day one — retrofitting idempotency after a duplicate corrupts a downstream store is painful. Adopt a schema registry with backward compatibility before you have your second consumer, so an additive migration never becomes an outage.

For any service that publishes events, use the outbox pattern rather than dual writes; it is the difference between “eventually consistent” and “occasionally wrong.” Treat the replication slot as a production database object with its own runbook, monitoring, and decommission checklist. And wire the dead-letter queue and its alerting on day one, not after the first poison message freezes a partition.

A pre-production checklist:

• [ ] Slot/binlog lag monitored, with alerts firing well before disk or retention limits.
• [ ] max_slot_wal_keep_size (Postgres) or adequate binlog_expire_logs_seconds (MySQL) set as a safety valve.
• [ ] Schema registry with backward compatibility; converters produce Avro/Protobuf, not raw JSON.
• [ ] Every consumer idempotent (upsert by key or dedupe by event ID).
• [ ] Dead-letter queue configured and alerted; replay procedure documented.
• [ ] Incremental snapshot enabled; ad-hoc snapshot signaling tested.
• [ ] Schema-history topic (MySQL) protected: single partition, infinite retention, replicated.
• [ ] Connector granularity chosen by change-rate and blast radius, not convenience.
• [ ] Slot/connector decommission steps written down so no orphan slot survives a deleted connector.

Done well, CDC becomes invisible infrastructure — the layer that keeps your warehouse, search index, cache, and downstream services agreeing with your operational database without a single dual write. The elegance is that you are not building a new source of truth. You are reading the one the database already keeps.

Frequently Asked Questions

What is the difference between log-based, query-based, and trigger-based CDC?

Query-based CDC polls tables on a timestamp or version column; it is simple but misses deletes, misses intermediate states between polls, and adds query load to live tables. Trigger-based CDC fires database triggers to write changes to a shadow table; it captures deletes but adds synchronous write amplification to every transaction and couples your schema to trigger logic. Log-based CDC reads the transaction log the database already writes, capturing every committed change in order with minimal overhead and full delete support. Log-based wins for completeness, ordering, and low impact, which is why Debezium uses it.

Does Debezium guarantee exactly-once delivery?

By default Debezium on Kafka Connect delivers at-least-once, meaning you may see duplicate events after a crash and restart, but you will not lose committed changes. Kafka’s KIP-618 adds exactly-once production into Kafka for source connectors, and Debezium has moved toward supporting it, but that guarantee ends at Kafka’s boundary. End-to-end exactly-once still requires idempotent consumers — typically upserting by primary key or deduplicating by event ID — because delivery to external sinks over non-transactional protocols can always retry.

How does the replication slot cause a Postgres outage?

A logical replication slot pins the write-ahead log at the position your connector last confirmed, so Postgres cannot recycle those WAL segments until the connector reads past them. If the connector dies and stays down, WAL accumulates continuously — tens of gigabytes per day under moderate load — until the pg_wal volume fills and the primary crashes. Monitor slot lag via pg_replication_slots, alert early, set max_slot_wal_keep_size as a safety valve, and always drop the slot when you remove a connector.

What is the outbox pattern and why do I need it?

The transactional outbox pattern solves the dual-write problem: instead of a service writing to its database and then separately publishing an event (which can partially fail), it writes an event row into an outbox table inside the same local transaction as the business change. Both commit atomically, so they can never disagree. Debezium reads the outbox row from the transaction log and its outbox-router transform publishes a clean domain event to Kafka. You get reliable event publishing without distributed transactions.

How does Debezium handle schema changes without breaking consumers?

Debezium pairs with a schema registry that stores each event schema and enforces a compatibility policy, usually backward compatibility. Additive, compatible changes such as a new nullable column register a new schema version and flow through; consumers that do not know the column ignore it. Breaking changes such as dropping a required column or narrowing a type are rejected at the boundary, so a bad migration fails loudly instead of silently poisoning downstream stores. On MySQL, Debezium also keeps a durable schema-history topic so it can interpret older log positions.

What is an incremental snapshot and when should I use one?

An initial snapshot reads a table’s full contents once at connector startup, which can block or take hours on large tables. An incremental snapshot, based on Debezium’s DDD-3 design, reads the table in configurable chunks while change streaming continues concurrently, using watermarks to deduplicate rows that change mid-snapshot. Use incremental snapshots for large tables, and use ad-hoc snapshots — triggered by sending a signal to a signaling table — to backfill a newly added table or re-seed a downstream store without restarting the connector or replaying the whole log.

Further Reading

• Iceberg vs Delta vs Hudi: lakehouse table formats ADR (2026) — where your CDC stream typically lands for analytics, and how the table format handles the constant upserts and deletes that change data capture produces.
• Async processing architecture patterns — the idempotency, retry, and consumer-design discipline that makes at-least-once CDC pipelines safe end to end.
• Apache Kafka tiered storage: KIP-405 architecture — how to retain CDC topics affordably for long replay windows without blowing up broker disk.
• Debezium documentation — the authoritative reference for connectors, snapshots, signaling, and transforms.
• PostgreSQL logical replication documentation — the primary source for slots, pgoutput, and wal_level configuration underpinning Postgres CDC.

By Riju — about