Real-Time Payment Infrastructure Engineering: Event-Driven Architecture Behind FedNow, UPI, and SEPA Instant

Last Updated: October 2026

Real-time payment infrastructure entered October 2026 with higher ceilings, heavier volumes and stricter name checks than our April article described. FedNow and the RTP network both allow transfers up to $10 million, UPI is processing more than 800 million transactions a day, and euro-area banks have been required to send instant payments and offer Verification of Payee since 9 October 2025.

The original article below is preserved exactly as published in April 2026. This update adds a dated section on what has moved, flags statements in the original that no longer hold or that we could not verify, and closes with a fresh FAQ and sources.

Disclaimer: this is a systems and architecture analysis for educational purposes, not financial, legal or investment advice.

What this covers: the October 2026 changes first, then the original deep dive, then an updated FAQ and sources.

What changed for October 2026

FedNow: $10 million limit and fast volume growth. The Federal Reserve raised the FedNow network limit for customer credit transfers and payment returns from $1 million to $10 million, and for liquidity management transfers from $2.5 million to $10 million, effective 12 November 2025. Participants can adopt the maximum or set lower limits, so the network ceiling is not the same as what your bank offers. The Fed’s autumn 2025 announcements also cited more than 1,500 participants, a pilot letting institutions verify receiver accounts before sending, integration of the Fed’s ScamClassifier model into fraud reporting, and an exception resolution service for disputes and information requests. On volume, the Fed’s published statistics show 4,997,811 settled payments in Q2 2026, up 83.2% from 2,728,510 in Q1 2026 and 2,466,494 in Q4 2025. ICBA, citing the Fed, reports average daily settled payments rising from 30,317 in Q1 to 54,921 in Q2. For architects, the lesson is that ten-million-dollar limits shift risk controls from per-payment caps toward real-time liquidity, fraud scoring and receiver verification. Dollar values on the Fed page looked internally inconsistent when we checked them, so we do not quote them here.

RTP network: $10 million limit, record days. The Clearing House’s RTP network raised its per-transaction limit to $10 million in early 2025 and now reports a single-day record of 2.27 million transactions worth $8.62 billion on 1 May 2026. It says it handles over 98% of US bank-to-bank instant payments, more than 1.5 million transactions on a typical day, and value approaching $500 billion per quarter. Its own release attributes part of the May spike to tax refund disbursements, which rose 78% in the first four months of 2026 versus 2025. These are operator-reported figures. The practical point for the settlement section below: two US rails now share the same $10 million ceiling, so multi-rail routing logic should compare cost, reach and cut-off behavior rather than amount limits.

UPI: 24 billion transactions a month, and the first merchant charge. NPCI data reported by DD News show UPI at 24.07 billion transactions in September 2026, up 23% year on year, worth Rs 29.37 lakh crore (up 18%), averaging about 802 million transactions a day. That sits well above the “14+ billion per month” and “450M+ daily” figures in our April text. The bigger rule change is a merchant discount rate (MDR) of 0.4% on person-to-merchant payments above Rs 2,000, starting 15 October 2026, with a Rs 300 cap on payments of Rs 75,000 and above. Press coverage says person-to-person transfers and merchant payments of Rs 2,000 or less stay free, that railways, telecom, insurance and fuel pay a flat Rs 5, and that capital-markets payments pay 0.02%. Consumers are not charged directly. Reports differ on whether the Finance Ministry or NPCI issued the framework, so check the primary notification before building billing logic around it.

SEPA Instant and Verification of Payee: the October 2025 deadline passed. Under the EU Instant Payments Regulation, euro-area payment service providers had to receive instant credit transfers from 9 January 2025, and to send them and offer Verification of Payee (VoP) from 9 October 2025. Non-euro member states follow on 9 January 2027 (receiving and price parity) and 9 July 2027 (sending and VoP). Instant transfers must be priced no higher than standard transfers, and the scheme-level amount cap was removed. Receiving banks must credit or reject within ten seconds. VoP is a name-matching check returning match, close match, no match or other, and must be offered free of charge for standard and instant transfers. A mid-2026 industry guide reports 2,792 PSPs in SCT Inst (79% of SCT participants) and instant payments at 30.7% of euro credit transfer volume in Q3 2025, up from 19.7% a year earlier. Instead of per-payment screening, providers must check their own customers against EU sanctions lists at least daily.

What this means for builders.

Design for limits as configuration. FedNow and RTP both allow $10 million, but each participating bank may set lower caps. Treat limits as per-counterparty data, not constants.
Add a payee-verification step. VoP in the euro area and the FedNow receiver-verification pilot both put an extra lookup before the payment event is emitted. Budget latency for it inside the ten-second window.
Expect fee logic to change. UPI’s MDR from 15 October means merchant-side services need amount-banded fee calculation and category exemptions.
Plan for volume step changes. FedNow’s 83% quarter-on-quarter jump and RTP’s holiday-style peaks argue for load-testing event pipelines at several times your current peak.

Statements in the April text to treat with care

“14+ billion transactions/month” and “450M+ daily” for UPI are now out of date; September 2026 was 24.07 billion and about 802 million a day.
The FedNow cost line ($0.01-$0.05 per transaction) and throughput figures such as 100K-200K msg/sec, and the UPI “9 aggregators” and 1M+ msg/sec figures, are unsourced in the original; we could not verify them.
The description of UPI settlement as hourly batches settled via NEFT next day, and of SEPA Instant clearing via “TARGET Instant by ECB” operated by the “Euro Bankers’ Association”, should not be relied on. The scheme is run by the European Payments Council, and infrastructure details need checking against EPC and ECB documentation.
Daily-volume cells in the comparison table (for example FedNow “~400K TX (~$30-50B)”) are unsourced estimates and are superseded by the figures above.
“Project MARQUES” as a World Bank initiative could not be verified.
“Last Updated: 2026-04-16” at the bottom of the original body is superseded by this update.

This article is for educational purposes only and does not constitute financial advice.

Disclaimer: This article is for educational purposes only and does not constitute financial advice.

Introduction

Real-time payments are no longer a technical frontier—they are an operational imperative. As of 2026, UPI processes over 14 billion transactions per month, the U.S. Federal Reserve’s FedNow has cleared over $2 trillion in aggregate volume, and SEPA Instant SCT Inst has become the standard for eurozone retail payments. The engineering underneath is a masterclass in distributed systems: sub-second latency requirements, exactly-once semantics in an asynchronous world, fraud detection in 500-millisecond windows, and near-zero tolerance for data loss.

This article dissects the event-driven architecture powering these systems. Unlike traditional request-response payment processing (HTTP POST, wait for response), modern real-time payment infrastructure treats every payment as an immutable event flowing through a topology of specialized, stateless processors. A single transaction triggers fraud analysis, settlement orchestration, reconciliation staging, and audit logging—all in parallel, with guaranteed delivery even if services fail.

We will examine the ISO 20022 message standards underpinning global payment flows, the settlement mechanics (instant gross settlement vs. deferred net settlement), streaming ML fraud detection in sub-second latency budgets, idempotency patterns for exactly-once delivery without distributed consensus, and reconciliation architectures that handle edge cases at scale. By the end, you will understand why Kafka/Pulsar topologies are preferable to traditional request-response APIs for payments, and the engineering trade-offs that define FedNow, UPI, and SEPA Instant.

1. Foundation: Why Event-Driven, Not Request-Response?

1.1 The Classical Problem: Synchronous Payment Processing

In a traditional synchronous architecture, a payment request follows a single call chain:

Client → Payment API → Fraud Check → Settlement Service → Ledger → Response

This architecture has three critical flaws for real-time payments:

Latency Coupling: Fraud detection, settlement orchestration, and audit logging all run in the critical path. If any service is slow, the entire payment is slow. A single 100ms outlier blocks the entire transaction chain.
Cascading Failure: If the Settlement Service is down, the entire payment gateway fails. There is no graceful degradation.
Loss During Failure: If a process crashes mid-transaction (e.g., fraud detector checks passed but settlement service crashed before responding), the client does not know whether the payment succeeded. Retrying is unsafe without idempotency.

Real-time payment schemes demand asynchronous decoupling of these concerns.

1.2 Event-Driven: Decoupling Concerns, Enabling Scale

Event-driven architectures invert the dependency flow:

Payment Initiation generates a single immutable event: PaymentInitiated(txID, amount, originator, beneficiary, timestamp).
Multiple independent consumers subscribe to this event and process it at their own pace:
– Fraud Detector consumes → outputs FraudDecision(txID, score, action)
– Settlement Processor consumes → outputs SettlementOrder(txID, status)
– Reconciliation Engine consumes → stages transaction for EOD matching
– Compliance Logger consumes → writes to audit trail
All consumers write back to the event stream: FraudDecision and SettlementOrder events become input to other consumers (e.g., if fraud score is high, settlement may be held).
No synchronous wait: The initiating client receives a receipt within milliseconds—not because all processing is done, but because the event is durable and ordered.

This architecture provides:
– Independent scaling: Add fraud processors without touching settlement infrastructure.
– Fault isolation: Settlement service crash does not block fraud detection.
– Durability: Events are persisted; no payment is lost if a processor crashes.
– Auditability: Event log is immutable; every decision is traceable.

FedNow, UPI, and SEPA Instant all rely on this topology.

2. Message Standards: ISO 20022 and the Grammar of Payments

2.1 ISO 20022: Structured Financial Message Exchange

ISO 20022 is the global standard for structured financial messages. Unlike unstructured HTTP APIs, ISO 20022 defines schemas, message types, and validation rules that apply across FedNow, UPI, SEPA Instant, and legacy systems.

Key message types in a real-time payment flow:

Message Type	Direction	Purpose	Example Fields
pain.001	Originator → Clearing House	Payment Initiation	debtor, creditor, amount, reference
pacs.008	Clearing House → Beneficiary Bank	Credit Transfer	instructedAmount, beneficiaryAccount, instructionID
pacs.002	Any Bank → Requestor	Payment Status Report	paymentInformationStatus (ACCP, ACSC, ACTC, RJCT)
pacs.004	Beneficiary → Originator	Positive Customer Credit Transfer Response	transactionIdentification, creditTransferResponseCode
pacs.028	Requestor → Any Bank	FIToFI Payment Status Inquiry	transactionIdentification

2.2 Terminology: Understanding Payment States

Instructed vs. Settled:
– Instructed: A payment order has been submitted and queued for processing. The amount is reserved but not yet debited.
– Settled: Funds have been moved from debtor account to creditor account. Settlement is irreversible (or reversible only by a separate reversal).

Gross vs. Net Settlement:
– Gross Settlement: Each transaction is settled individually and immediately. If Bank A sends $10 to Bank B, and Bank B sends $3 to Bank A, two separate transfers occur. Used by FedNow and SEPA Instant.
– Net Settlement: Multiple transactions are aggregated, and only the net difference is settled. If Bank A owes Bank B $10 and Bank B owes Bank A $3, only one transfer of $7 from A to B occurs. Used by legacy ACH, reduces reserve requirements but adds delay (T+1 to T+2).

Idempotency Key / Correlation ID:
– A unique client reference (e.g., ClientRef: TX-2026-04-16-001) that identifies a payment even if retransmitted. If the same ClientRef is submitted twice, the system processes it only once.

2.3 Diagram: Message Flow in Real-Time Payment

This diagram shows:
1. Originator Bank sends pain.001 (payment initiation) with debtor details.
2. Clearing House / Central Bank translates to pacs.008 and routes to the Payment Switch.
3. Payment Switch enqueues PaymentInitiated event into the event stream.
4. Fraud Detector subscribes and emits FraudDecision.
5. Settlement Engine subscribes to approved payments and executes debit/credit.
6. Beneficiary Bank receives funds and sends pacs.002 + pacs.004.
7. Idempotency Cache (Redis/DynamoDB) deduplicates retries and ensures exactly-once delivery.

3. Event-Driven Architecture: From Kafka/Pulsar to Business Logic

3.1 Event Stream Topology

Real-time payment systems use a distributed message broker (Kafka or Apache Pulsar) as the backbone. The topology is a directed acyclic graph of topics (event streams) and consumer groups (processors).

Primary Topic: payments-topic
– Throughput: 100K-1M events per second (FedNow peak: ~200K msg/sec, UPI: ~160K msg/sec).
– Message schema (Avro/Protobuf):
{ txID: "TXID-99234", clientRef: "TX-2026-04-16-001", // Idempotency key timestamp: 1713282345123, // Event creation (ns precision) originator: { bankCode: "FRB", accountID: "ACC-123-456" }, beneficiary: { bankCode: "JPM", accountID: "ACC-456-789", name: "Jane Doe" }, amount: 100.00, currency: "USD", channel: "mobile", // For fraud scoring ipAddress: "203.0.113.45", deviceID: "device-hash-xyz", narration: "Invoice Payment" }

Derived Topics (fed by consumer groups):
– fraud-alerts-topic: High-risk transactions flagged for challenge.
– settlement-events-topic: Approved transactions ready for ledger posting.
– reconciliation-topic: Transactions staged for end-of-day matching.
– dead-letter-queue: Messages that failed processing (for manual review).

3.2 Consumer Groups and Parallelism

A consumer group is a set of parallel processes that jointly process a topic, with partition assignment ensuring each message is processed exactly once by one instance.

Example: Fraud Detection Consumer Group
– 16 instances, each assigned 4-8 partitions.
– Each instance maintains a state store (RocksDB) of:
– 5-minute rolling aggregate of transactions per account.
– Device fingerprint baseline for the account holder.
– Real-time feature vector (velocity, geographic anomaly, network risk).
– Processing latency: 10-50ms per message (inference included).
– Window: First fraud decision is made within 500ms of event receipt; can be extended to 1-2 seconds for real-time ML scoring.

Example: Settlement Processor Consumer Group
– 8 instances, each assigned 6-8 partitions.
– Aggregates payments by (originator, beneficiary) pair over a 1-100ms window.
– Writes aggregated settlement order to settlement-events-topic.
– Maintains state: current settlement batch, amounts by account pair.

3.3 Diagram: Event-Driven Topology

This diagram illustrates:
1. Event Sources: Payment API, batch uploads, mobile channels all write to payments-topic.
2. Event Stream: payments-topic is the source of truth; events are persisted with replication factor 3.
3. Consumer Groups (Fraud Detector, Settlement Processor, Reconciliation Engine, Compliance Logger) subscribe independently.
4. Feature Enrichment: Fraud Detector queries a real-time feature store (RocksDB state store or external DB).
5. Feedback Loop: Fraud and settlement decisions are emitted back to the topic for downstream consumers.
6. Data Layer: State stores (RocksDB), feature databases, ledger, and reconciliation tables capture state.

3.4 Why Kafka/Pulsar Over Direct Service-to-Service?

Consider an alternative: Fraud Detector directly calls Settlement Processor via HTTP. Immediately, the architecture fails:
– If Fraud Detector is slow, Settlement is blocked (latency coupling).
– If Fraud Detector crashes mid-call, Settlement does not know if the decision was made (loss risk).
– Adding a new consumer (e.g., Blockchain audit logger) requires changing Fraud Detector’s code.

With Kafka:
– Fraud Detector writes to fraud-alerts-topic; any consumer can subscribe.
– Each consumer is independent; if one is slow or down, others are unaffected.
– Scaling is decoupled: add fraud detector instances without touching settlement.

4. Settlement Mechanics: Instant Gross vs. Deferred Net

4.1 Instant Gross Settlement (FedNow, SEPA Instant)

Model: Each transaction is settled individually, immediately, and irrevocably.

Timeline:
1. Payment event is received (T+0ms).
2. Fraud check clears within 500ms.
3. Funds are debited from originator’s account (T+100-500ms).
4. Funds are credited to beneficiary’s account (T+200-800ms).
5. Both parties receive confirmation within 1-2 seconds.

Ledger Posting:
– Originator bank: Debit to Payment Account, Credit to Settlement Account (prefunded).
– Central Bank: Debit to Originator Settlement Account, Credit to Beneficiary Settlement Account (RTGS—Real-Time Gross Settlement).
– Beneficiary bank: Debit to Settlement Account, Credit to Payment Account.

Liquidity Requirement: Banks must maintain prefunded settlement accounts at the central bank. For FedNow, the average prefunded balance is 2-5% of daily payment volume. For a bank processing $1B/day, that is $20-50M in reserve at the Fed.

Benefit: Funds reach beneficiary within seconds; finality is immediate.

Cost: High reserve requirement (opportunity cost), plus intraday overdraft facility fees if a bank’s prefunded account runs low.

4.2 Deferred Net Settlement (Legacy ACH, Wire Clearing)

Model: Transactions are aggregated over a settlement period (60 minutes to 1 day), then only the net difference between counterparties is settled.

Timeline:
1. Payment event is received (T+0min).
2. Transactions are queued in a settlement batch (T+0 to T+60min).
3. Batch closes; net positions are calculated (T+60min).
4. Settlement occurs (T+120min to T+next-business-day).
5. Confirmation is delayed by settlement window (T+1 to T+3).

Ledger Posting:
– Originator bank: Memo debit immediately; actual debit after settlement.
– Central Bank: Batch net debit/credit (one transaction per bank pair).
– Beneficiary bank: Memo credit immediately; actual credit after settlement.

Liquidity Requirement: Lower because only net amounts are settled. A bank owing $1M and being owed $0.8M settles $0.2M, not $1M. Reserve requirements are ~0.5% of daily volume.

Benefit: Lower reserve burden, lower cost for high-volume corridors.

Cost: Delayed finality (funds take 1-3 days); higher settlement risk if a bank fails mid-settlement period.

4.3 Diagram: Settlement Mechanics

This diagram contrasts:
– Left side (Instant): FedNow and SEPA Instant follow immediate gross settlement. No batching, no netting.
– Right side (Deferred): Legacy ACH batches over 1 hour, aggregates transactions, nets them, and settles at T+1.
– Liquidity Box: Both models require prefunded settlement accounts; instant settlement requires larger reserves.

4.4 Scale Numbers: Reserve and Throughput

FedNow (U.S. Federal Reserve, 2024-2026):
– Participating banks: 9,000+
– Daily transaction volume: ~400K transactions (growing; peak observed: 1.2M).
– Average transaction size: $50-200K (median: ~$20K).
– Aggregate daily volume: ~$30-50B.
– Total FedNow settled volume to date: $2+ trillion (cumulative).
– Prefunded settlement account average: $100M-500M per bank (varies by size).

UPI (NPCI, India):
– Monthly transaction count: 14+ billion (2026 projection).
– Daily: ~450M transactions.
– Average transaction size: ₹500-2000 (USD 6-25).
– Aggregate daily volume: ₹2-3 trillion (USD 24-36 billion).
– Processors: NPCI operates 9 regional payment systems; peak throughput: 1M+ msg/sec across network.

SEPA Instant (EBA, Europe):
– Monthly transactions: ~200M+ (growing).
– Daily: ~6-7M transactions.
– Average transaction size: €50-500.
– Aggregate daily volume: €20-30B.
– Participating banks: 3,000+.

5. Fraud Detection in Sub-Second Windows

5.1 Streaming ML: Real-Time Feature Extraction

Traditional fraud detection is batch: analyze 10,000 transactions every night, flag suspicious ones, block them next morning. This does not work for real-time payments; fraudsters exploit the detection lag.

Streaming fraud detection operates on each transaction as it arrives, with decisions made in 10-500ms. This requires:

Real-Time Features:
– Velocity: How many transactions in the last 5 minutes? How much value? (Threshold: > 10 TX in 5min or > $10K/5min → risk score +20).
– Device Fingerprint Deviation: Is this device new or returning? (New device + high amount → risk score +30).
– Geographic Anomaly: Is the transaction originating from an impossible location? (Last transaction in NYC 2 minutes ago, now from Singapore → risk score +40).
– Network Risk: Is the beneficiary account a known fraud sink? (Graph-based: analyze payment network for clusters of compromised accounts).
– Behavioral Baseline: What is the account holder’s usual transaction pattern? (Time of day, amount range, recurring beneficiaries).
Inference Latency Budget:
– Feature extraction: 5-10ms.
– Rule engine (thresholds): 1-5ms.
– ML model inference (XGBoost/neural net): 10-50ms (or async, if < 500ms acceptable).
– Total: 16-65ms from event receipt to decision.
Handling Latency Spikes:
– If inference takes > 500ms, the transaction is challenged (OTP/2FA sent to customer’s phone).
– Challenge can extend detection window to 60 seconds.
– If no response, transaction is held pending manual review (post-transaction monitoring).

5.2 Streaming ML Architecture

Component 1: State Store (RocksDB / Druid)
– Stores 5-minute rolling aggregates, device fingerprints, and baseline profiles.
– Updated as events arrive; old windows are evicted (LRU).
– Latency: <1ms per query (in-process for RocksDB, <10ms for remote Druid).

Component 2: Feature Enrichment
– For each incoming payment event, fetch:
– Account holder’s baseline (peak spending hour, usual amount range).
– Device fingerprint database (is device registered to this account?).
– 5-minute velocity (count, sum).
– Beneficiary reputation (is it a known fraud sink?).

Component 3: Rule Engine + ML Model
– Rules (deterministic): Apply thresholds.
– Example: velocity_5min > 10 && amount > 5000 → score += 30.
– ML Model (probabilistic): XGBoost or Neural Net trained on labeled fraud.
– Input: feature vector (velocity, device deviation, geographic anomaly, network risk, etc.).
– Output: fraud probability (0-100).
– Retrained daily with the previous day’s labeled examples (chargeback signals, manual reviews).

Component 4: Decision
– Risk score < 30: Accept (< 1% fraud rate).
– Risk score 30-70: Challenge (Ask for OTP/2FA; ~10% fraud rate if challenged).
– Risk score > 70: Decline + alert fraud team (> 30% fraud rate).

5.3 Diagram: Fraud Detection in Sub-Second Windows

This diagram shows:
1. Input: PaymentInitiated event with metadata (amount, beneficiary, device, IP, etc.).
2. Feature Extraction: Query RocksDB state store for velocity, device fingerprint, geographic baseline.
3. Enrichment: Fetch behavioral baseline and network risk from feature databases.
4. Detection: Rule engine + ML model both score; max or ensemble score is used.
5. Decision: Split on risk thresholds.
6. Feedback Loop: Chargeback signals (T+30 days) feed into daily model retraining.

5.4 Handling Post-Transaction Fraud

Challenge: Real-time fraud detection is imperfect. Some fraudulent transactions slip through (false negatives); some legitimate ones are incorrectly flagged (false positives).

Post-Transaction Monitoring (T+30 days):
– Chargeback signals arrive: “This transaction was unauthorized.”
– Label this transaction as fraud = true in historical database.
– Retrain ML model (daily batch job, 1-2 hour latency).
– Next day, similar transactions are scored higher.

Reversal:
– If a transaction is identified as fraud after settlement, it is reversed (pacs.007 or pacs.180 message sent).
– Originator account is credited; beneficiary account is debited.
– Reconciliation system detects mismatch and flags for investigation.

6. Exactly-Once Delivery and Idempotency

6.1 The Problem: Duplicates in Distributed Systems

Networks fail. A client sends a payment request; the server processes it and returns a response, but the response is lost. The client retries. Now the same payment may be processed twice.

Without idempotency: Bank Account balance goes:

Initial: $1000
TX-1: $100 sent → $900
Retry (duplicate): $100 sent again → $800
Final: $800 (WRONG; should be $900)

With idempotency: Bank Account balance goes:

Initial: $1000
TX-1 (ClientRef: TX-001): $100 sent → $900; cache[TX-001] = {txID: TXID-99234, status: SETTLED}
Retry (ClientRef: TX-001): Found in cache; return cached response → $900
Final: $900 (CORRECT)

6.2 Idempotency Architecture

Key Components:

Idempotency Key (Client-provided): ClientRef: "TX-2026-04-16-001". Unique per originator+time+intent.
Idempotency Cache (Server-side):
– Store: {ClientRef → (txID, status, timestamp, response)}
– Backing: Redis (for speed) or DynamoDB (for durability).
– TTL: 24 hours (covers retry window + regulatory retention).
– Write-once: First write wins; subsequent writes are rejected.
Transactional Payment Execution:
– Atomically: (1) check cache, (2) if not found, execute payment, (3) write cache, (4) return response.
– Database transaction or compare-and-swap semantics ensure atomicity.
Idempotency Key Scope:
– Global: Unique across all clients. Safe but requires global coordination (Zookeeper, Raft consensus). Slow.
– Per-Originator: Unique per bank. Simple; no global coordination needed.

6.3 Idempotency + Event Stream

Challenge: If payment execution is deferred (enqueued to event stream), how do we know idempotency is honored?

Solution: Write idempotency record before enqueuing event.

1. Client: POST /payments with ClientRef: TX-001, Amount: 100
2. Server:
   a. Acquire lock on (ClientRef)
   b. Check cache: not found
   c. Generate txID: TXID-99234
   d. Write idempotency record: {ClientRef: TX-001, txID: TXID-99234, status: PENDING, timestamp: ...}
   e. Enqueue PaymentInitiated(txID: TXID-99234, ...) to event stream
   f. Release lock
   g. Return: {txID: TXID-99234, status: PENDING, clientRef: TX-001}
3. Fraud detector processes event; emits FraudDecision
4. Settlement processor processes event; posts to ledger; updates cache to status: SETTLED
5. Client retries (network timeout):
   a. Server: Check cache; found with status: PENDING or SETTLED
   b. Return cached response (no re-execution)

Guarantee: Even if the event stream duplicates the message (e.g., Kafka rebalance), the settlement processor’s consumer group ensures it is only processed once per partition-offset.

6.4 Diagram: Idempotency + Exactly-Once

This diagram shows:
1. Request: Client sends POST with ClientRef (idempotency key).
2. Cache Check: Query idempotency store (Redis/DynamoDB).
3. Cache Miss: Execute payment, write cache, return.
4. Cache Hit: Replay; return cached response. (If status is PENDING, may re-execute or wait; if SETTLED, return final response.)

7. Reconciliation at Scale

7.1 The Challenge: End-of-Day Matching

Every transaction has two sides: an instruction (originator bank’s record: “we sent $100”) and a settlement (beneficiary bank’s record: “we received $100”). These must match.

Daily Reconciliation:
– Central Bank publishes settlement file (ISO 20022 pacs.002 messages).
– Each bank downloads file; loads into reconciliation engine.
– Engine matches instructed transactions with settlement confirmations (many-to-many; e.g., $100 + $50 instruct may settle as $150 together).
– Mismatches are flagged: unmatched instructed (missing settlement), unmatched settlement (no instruction), amount mismatch.

Scale: FedNow reconciles ~400K transactions daily. UPI reconciles ~450M transactions daily.

7.2 Matching Algorithm

Input:
– Instructed transactions: {txID, amount, originator, beneficiary, timestamp}
– Settlement confirmations: {settlementID, amount, originator, beneficiary, timestamp}

Matching Strategy:
1. Exact Match: Same txID, amount, pair. Mark as matched.
2. Amount Match: Aggregated set of instructions (sum) matches aggregated set of settlements (sum) for a pair.
– Example: Instruct [100, 50] and Settle [150] for (Bank A, Bank B) → match.
3. Fuzzy Match (if enabled): Allow for rounding errors (e.g., ±0.01 on large amounts).

Algorithm Complexity: O(n^2) brute-force is infeasible for millions of transactions. Real systems use:
– Hash-based grouping: Group by (originator, beneficiary) pair; within each group, run matching.
– Time-window grouping: Transactions within 1-2 hour window may be aggregated.
– Two-phase approach: Phase 1 (deterministic), phase 2 (ML-assisted fuzzy matching).

7.3 Exception Handling

Unmatched Instructed:
– Originator instructed a payment, but no settlement confirmation received.
– Cause: System failure (payment lost in transmission), beneficiary bank offline, incorrect beneficiary account.
– Resolution:
– Issue status inquiry (pacs.028) to central bank or beneficiary bank.
– If no response within 5 business days, auto-reverse (debit reversal, pacs.180).
– Manual investigation team follows up on reversals.

Unmatched Settlement:
– Beneficiary bank received funds, but no matching instruction.
– Cause: Instructed transaction was not recorded by originator bank (system crash), or settlement is duplicate.
– Resolution:
– Cross-bank investigation (beneficiary bank provides settlement evidence).
– If fraudulent, reversal initiated; beneficiary account debited.
– If erroneous duplicate, originator bank reverses their duplicate instruction.

Amount Mismatch:
– Instruction: $100. Settlement: $99.50 (rounding, exchange rate).
– Cause: FX conversion rounding, fees deducted.
– Resolution:
– If within tolerance (0.1% or $0.01), auto-reconcile.
– If outside tolerance, manual review; both banks must agree on reconciliation or reversal.

7.4 Reconciliation State Machine

Status Transitions:
– Instructed: Initial state; awaiting settlement.
– Pending: Settlement received, but not yet matched (within grace period, e.g., 24 hours).
– Reconciled: Matched; settled in general ledger.
– Exception: Unmatched after grace period; flagged for investigation.
– Reversed: Originator or beneficiary initiated reversal; amount is returned.

7.5 Diagram: Reconciliation Process

This diagram shows:
1. Settlement Day: Instruction is recorded in ledger.
2. EOD Monitoring: Events stream and settlement file are loaded.
3. Matching Engine: Reconciliation algorithm matches instructed to settlement.
4. Outcomes: All matched → reconciled. Mismatches → exceptions.
5. Exception Handling: Status inquiry, manual review, auto-reversal.
6. Final State: All transactions are reconciled and settled in GL by T+1 or T+2.

8. System Design: FedNow, UPI, and SEPA Instant

8.1 FedNow (U.S. Federal Reserve)

Architecture Overview:

Initiation: Banks connect via secure SFTP or HTTP APIs to FedNow portal. Messages are ISO 20022 pain.001.
Central Processor: Federal Reserve operates FedNow, a centralized payment switch.
Validates pain.001; generates pacs.008.
Enqueues to internal event stream (custom Kafka-like system, high reliability).
Processes 100K-200K msg/sec (peak).
Fraud Module: FedNow includes basic fraud checks (blacklisting, limit checks). Individual banks add their own ML-based fraud detection.
Settlement: Gross settlement in Federal Reserve funds (FEDWIRE). Central bank debits originator’s RTA (Reserve Account at The Fed), credits beneficiary bank’s RTA.
Confirmation: pacs.002 + pacs.004 returned within 1-2 seconds.
Reconciliation: Daily batch; Fed publishes settlement file. Banks reconcile against their records.

Latency SLA: < 2 seconds end-to-end (initiation to beneficiary account credit).

Availability: 24/7/365 operation (only down for brief maintenance windows).

Cost: Flat per-transaction fee ($0.01-$0.05 per transaction, varies by bank size and volume commitment).

8.2 UPI (NPCI, India)

Architecture Overview:

Initiation: Mobile app or USSD (feature phone). Messages are custom NPCI format (XML-based, similar to ISO 20022 semantics).
NPCI Aggregators: 9 nodal processors (banks or telecom companies) operate regional switches.
Each aggregator processes ~50K msg/sec (peak across network: 1M+ msg/sec).
Aggregators are connected in a mesh; messages are routed by PSP (Payment Service Provider) ID.
Fraud Module: NPCI has central fraud engine; each bank also runs local fraud detection.
Settlement: Net settlement model (deferred). Transactions are batched hourly and settled via NEFT (National Electronic Funds Transfer) next day.
Recent Change: NPCI introduced “TPAP” (Third-Party App Provider) support; banks can now process UPI within their own systems (reducing latency).
Confirmation: UPI app shows transaction status within 2-10 seconds (includes peer-to-peer confirmation).
Reconciliation: Banks reconcile against NPCI master settlement file daily.

Scale: 14+ billion transactions/month (as of 2026); 450M+ daily.

Latency SLA: < 10 seconds for peer-to-peer; real-time for bank account to UPI ID (TPAP).

Cost: Free for end users; nominal processing fee for merchants (0.5%-2%, negotiated).

8.3 SEPA Instant (EBA, Europe)

Architecture Overview:

Initiation: Banks send ISO 20022 pain.001 (Credit Transfer Initiation Message).
Central Infrastructure: EBA (Euro Bankers’ Association) operates SCT Inst scheme; participating banks connect via SWIFT network or proprietary links.
Central clearing house (e.g., TARGET Instant by ECB) processes transactions.
Throughput: 60K-100K msg/sec.
Fraud Module: Each bank responsible for fraud detection; EBA provides guidelines (ECB recommendations).
Settlement: Gross settlement via ECB’s RTGS (Real-Time Gross Settlement). Central bank debits originator’s account (in euros), credits beneficiary’s account in real-time.
Confirmation: pacs.002 (payment status) and pacs.004 (positive response) within 10 seconds.
Reconciliation: Banks reconcile daily; SWIFT provides settlement records.

Latency SLA: < 10 seconds (ECB target).

Availability: 24/7/365 (unlike legacy SEPA SCT, which is business-hours only).

Cost: Fixed annual fee per bank + per-transaction fee (~€0.001-0.01 per transaction).

8.4 Comparative Table

Aspect	FedNow	UPI	SEPA Instant
Settlement Model	Gross	Deferred Net (with TPAP instant option)	Gross
Clearing House	Federal Reserve	NPCI (9 aggregators)	ECB / EBA
Message Standard	ISO 20022	NPCI XML (ISO-like)	ISO 20022
Peak Throughput	200K msg/sec	1M+ msg/sec (network)	100K msg/sec
Latency SLA	< 2 sec	< 10 sec (< 2 sec with TPAP)	< 10 sec
Coverage	US	India + diaspora	Eurozone
Daily Volume	~400K TX (~$30-50B)	~450M TX (~$25-36B)	~6-7M TX (~$20-30B)
24/7 Operation	Yes	Yes	Yes
Fraud Detection	Fed + bank-level	NPCI + bank-level	Bank-level (ECB guidance)

9. Advanced Patterns and Challenges

9.1 Handling Late-Arriving Data

Scenario: A settlement confirmation arrives 2 days late due to network outage. Reconciliation has already completed and marked it as unmatched exception.

Solution: Reconciliation engine maintains a late-arrival grace window (2-5 days). If a late-arriving settlement is received within this window, the engine re-runs matching and updates status from “Exception” to “Reconciled”.

Implementation: Batch job runs daily; queries for recent exceptions; checks for late arrivals; updates ledger.

9.2 Cascading Reversals

Scenario: Payment A is reversed (unauthorized). Payment B depends on funds from A (e.g., originator uses received funds to pay a third party). Now B’s funding is in question.

Challenge: Event-driven architecture makes causality tracking difficult.

Solution: Causality ID (similar to trace ID in distributed tracing).
– Original payment A: causality_id: CAU-001
– Dependent payment B: causality_id: CAU-001 (inherited)
– Reversal of A triggers reversal of B (and B’s dependents, recursively).
– Causality graph is tracked in a separate service for audit.

9.3 Cross-Border Real-Time Payments

Challenge: Real-time payments require tight coupling between two central banks’ systems (e.g., FedNow ↔ SEPA Instant for US-Europe transfer).

Status: As of 2026, cross-border real-time payments are still experimental. Most cross-border transfers use SWIFT gpi (clearing in 1-2 hours) or legacy correspondent banking (2-5 days).

Barriers:
– Different settlement models (gross vs. net).
– Different fraud standards and thresholds.
– Exchange rate volatility and FX settlement lag.
– Regulatory approval (not all countries allow real-time interoperability).

Future: Project MARQUES (World Bank initiative) aims to link real-time payment systems by 2027-2030.

9.4 Consumer Confidence and Disputes

Problem: Real-time settlement is irreversible. If a consumer is defrauded or disputes a payment, the original sender’s bank cannot simply reverse it (unlike deferred settlement).

Solution: Real-time payment systems rely on fraud prevention (minimize unintended transfers) rather than reversal (undo after the fact).
– Strong authentication (2FA, biometric).
– Real-time fraud detection.
– Beneficiary account holder contact before funds arrive (optional; some schemes use this).

Chargeback / Dispute: If fraud still occurs, originator’s bank initiates a reversal request (pacs.180). Beneficiary bank must process within 5 business days (per regulations). If no agreement, dispute goes to network (Fed, EBA, NPCI) for arbitration.

10. Engineering Insights: Why Event-Driven Wins

10.1 Fault Isolation

Synchronous (request-response):

Payment API → Fraud Service → Settlement Service → Ledger
If Settlement crashes, entire pipeline fails.

Event-driven:

Payment API → Event Stream
                 ├→ Fraud Consumer Group (isolated)
                 ├→ Settlement Consumer Group (isolated)
                 └→ Reconciliation Consumer Group (isolated)
If Settlement Consumer crashes, Fraud and Reconciliation continue.

10.2 Independent Scaling

Synchronous: To handle a spike in fraud detection latency, you must upgrade the Fraud Service CPU. This may over-provision other services.

Event-driven: Add more Fraud Consumer instances. Settlement, Reconciliation, and others remain unchanged.

10.3 Durability

Synchronous: If Settlement Service crashes mid-operation, the client does not know if the payment succeeded. Retry is unsafe without idempotency.

Event-driven: Event is durable (persisted to event stream with replication). If Settlement Consumer crashes, Kafka re-assigns the message to another instance. No data loss.

10.4 Observability and Debugging

Synchronous: Stack trace shows the call chain; difficult to understand what happened to a specific transaction if multiple services are involved.

Event-driven: Event log is the source of truth. Trace a transaction:

1. Event: PaymentInitiated(txID-99234) at T+0ms
2. Event: FraudDecision(txID-99234, score=45, action=CHALLENGE) at T+120ms
3. Event: SettlementOrder(txID-99234, status=HELD) at T+200ms
4. Manual override: SettlementApproved(txID-99234) at T+5s
5. Event: PaymentSettled(txID-99234) at T+5.5s

Each step is logged and queryable. Debugging is a matter of examining event timestamps and outcomes.

10.5 Replay and Auditing

Event-driven: Entire transaction history is immutable. Regulators can replay events to audit (“show me every step that happened to this payment”).

Synchronous: Audit trail is a separate log; synchronization errors are possible.

11. Operational Considerations

11.1 Monitoring and Alerting

Key Metrics:
– Latency: P50, P95, P99 end-to-end (initiation to beneficiary credit).
– Throughput: Transactions per second, per minute, per hour.
– Error Rate: Percentage of transactions failing (target: < 0.01%).
– Fraud Rate: Percentage of transactions flagged as fraud (baseline: 0.1-0.5% depending on payment type and channel).
– Reconciliation Success Rate: Percentage of transactions matched on first pass (target: > 99.9%).

Alerts:
– Latency spike (P95 > 500ms).
– Error rate spike (> 0.1%).
– Consumer lag (event stream → processor lag > 1 minute).
– Dead letter queue size growth (unprocessable messages).

11.2 Disaster Recovery

RTO (Recovery Time Objective): < 5 minutes for critical services.

RPO (Recovery Point Objective): < 10 transactions (acceptable data loss if all else fails).

Strategy:
– Event stream replication factor: 3 (across 3 data centers or availability zones).
– Database replicas: Multi-primary (active-active) for settlement ledger.
– Backup: Daily snapshots of state stores; offsite storage.
– Failover: Automatic detection and rerouting of traffic to replica instances.

11.3 Compliance and Regulations

Standards:
– ISO 20022: Global standard for message format; mandatory for FedNow, SEPA Instant, UPI.
– PCI DSS: Payment Card Industry Data Security Standard; not directly applicable but similar principles apply.
– AML/CFT: Anti-Money Laundering / Counter Financing of Terrorism. Real-time systems must screen transactions against sanctions lists in < 1 second.

Audit:
– Transaction logs must be immutable (blockchain-inspired, but traditional databases with WAL are sufficient).
– Audit trail must cover origination, fraud checks, settlement, reconciliation, and reversals.
– Regulators have direct access to audit logs (remote audit or batch export).

12. Future Directions

12.1 Blockchain-Based Settlement

Opportunity: Central Bank Digital Currencies (CBDCs) are in pilot. A CBDC settlement system (e.g., digital dollar or digital euro) could replace RTGS with direct blockchain settlement.

Advantage: Instant settlement (no intermediary); transparent ledger; atomic swaps (pay and receive in a single transaction).

Challenge: Regulatory approval, technical complexity, integration with existing systems.

Timeline: 2028+ for production CBDCs.

12.2 Machine Learning for Fraud and Risk

Trend: Moving from rule-based fraud detection to deep learning models (transformer networks, graph neural networks for network risk).

Advantage: Better false positive / false negative trade-off; adapts to evolving fraud tactics.

Challenge: Model explainability (regulators require interpretable decisions); retraining latency (must incorporate chargeback signals daily).

12.3 Cross-Border Real-Time

Opportunity: Link real-time payment systems across countries (FedNow ↔ SEPA Instant ↔ UPI).

Challenge: Regulatory harmonization, currency conversion, risk management.

Timeline: 2027-2030 (Project MARQUES, ECB initiatives).

13. Conclusion

Real-time payment infrastructure is a masterclass in distributed systems engineering. The shift from synchronous request-response architectures to event-driven topologies enables:

Fault isolation: Consumer failures do not cascade.
Independent scaling: Add capacity to one service without affecting others.
Durability: Events are persisted; no transaction is lost.
Auditability: Immutable event log is the source of truth.

The engineering underpinning FedNow (200K msg/sec, sub-2-second latency), UPI (1M+ msg/sec, 450M daily transactions), and SEPA Instant (100K msg/sec, sub-10-second latency) shares common patterns:

ISO 20022 message standards for interoperability.
Kafka/Pulsar event streams for decoupling and durability.
Streaming ML fraud detection in 500ms windows.
Idempotency caches for exactly-once delivery.
Reconciliation engines for end-of-day matching with exception handling.
Multi-level settlement (instant gross or deferred net) based on regulatory requirements.

Understanding these patterns is essential for fintech engineers building payment systems, regulators designing infrastructure, and technologists anticipating the next generation of financial networks. The future likely involves cross-border real-time interoperability and CBDC-enabled atomic settlement, further raising the bar for latency, scale, and reliability.

References and Further Reading

ISO 20022: https://www.iso.org/standard/81090.html (official standard)
Federal Reserve, FedNow: https://www.frbservices.org/news/blogs/2023/may/fednow-service.html
NPCI, UPI: https://www.npci.org.in/what-we-do/upi/upi-system
EBA, SEPA Instant: https://www.ebanet.org/services/sepa-instant-credit-transfer-sct-inst
Apache Kafka: https://kafka.apache.org/ (event streaming platform)
Apache Pulsar: https://pulsar.apache.org/ (event streaming alternative)
ECB, Real-Time Gross Settlement: https://www.ecb.europa.eu/paym/target/html/index.en.html
SWIFT Standards: https://www.swift.com/standards (financial messaging standards)
Project MARQUES: World Bank initiative for cross-border real-time linkage (2026+ status).
Designing Data-Intensive Applications by Martin Kleppmann (recommended reading for event-driven systems).

Published: 2026-04-16
Author: iotdigitaltwinplm.com Editorial
Pillar: FinTech
Primary Keyword: real-time payment infrastructure engineering
Secondary Keywords: event-driven architecture, ISO 20022, FedNow, UPI, SEPA Instant, fraud detection, reconciliation, idempotency, exactly-once delivery

Updated FAQ (October 2026)

What is real-time payment infrastructure?

Real-time payment infrastructure is the set of messaging, clearing, settlement and risk systems that move money between accounts within seconds, around the clock. Typical building blocks are ISO 20022 style messages, event streams, idempotent processing, streaming fraud scoring and continuous reconciliation. Examples include FedNow and the RTP network in the United States, UPI in India and SEPA Instant in Europe. The key engineering difference from batch rails is that settlement finality, fraud checks and customer confirmation all happen inside one short window.

What is the FedNow transaction limit in October 2026?

The FedNow network limit for customer credit transfers and payment returns is $10 million, raised from $1 million effective 12 November 2025. Liquidity management transfers were raised from $2.5 million to $10 million at the same time. Individual participating institutions may choose lower limits, so your bank or customer may be capped well below the network maximum. Always confirm the limit your own financial institution applies before designing a high-value payment flow around it.

How big is UPI in 2026 and what changes on 15 October?

NPCI data reported by DD News put UPI at 24.07 billion transactions in September 2026, about 802 million a day, worth Rs 29.37 lakh crore. From 15 October 2026 a 0.4% merchant discount rate applies to person-to-merchant payments above Rs 2,000, capped at Rs 300, according to press coverage of the framework. Person-to-person transfers and smaller merchant payments remain free. Confirm details and exemptions against the official notification before changing billing systems.

What is Verification of Payee and when did it become mandatory?

Verification of Payee is a name-matching check performed before an euro credit transfer is sent. The payer’s bank asks the payee’s bank whether the name and account identifier match, and receives match, close match, no match or other. Euro-area providers had to offer it free of charge from 9 October 2025. Providers in non-euro EU states have until 9 July 2027. It is aimed at reducing misdirected payments and authorised push payment fraud.

How fast must SEPA Instant payments be?

Under the EU Instant Payments Regulation, the receiving provider must make funds available to the payee, or reject the transfer, within ten seconds of receiving the payment order. Euro-area providers had to send instant payments from 9 October 2025, following the receiving obligation from 9 January 2025. Instant transfers cannot cost more than standard transfers. Non-euro member states must receive from 9 January 2027 and send from 9 July 2027.

Is the RTP network limit the same as FedNow?

Yes at the network level. The Clearing House’s RTP network allows up to $10 million per transaction, and FedNow raised its network limit to $10 million in November 2025. Both operators allow participating banks to set lower limits, so what a customer can actually send depends on the bank. RTP reported a record day on 1 May 2026 with 2.27 million transactions worth $8.62 billion, which are operator-reported figures. Treat both ceilings as configuration, not constants, when you design routing and risk controls.

Sources and Further Reading

This post is a systems and architecture analysis for educational purposes and is not financial, legal or investment advice.

By Riju – about

Real-Time Payment Infrastructure Engineering: Event-Driven Architecture Behind FedNow, UPI, and SEPA Instant

What changed for October 2026

Statements in the April text to treat with care

Introduction

1. Foundation: Why Event-Driven, Not Request-Response?

1.1 The Classical Problem: Synchronous Payment Processing

1.2 Event-Driven: Decoupling Concerns, Enabling Scale

2. Message Standards: ISO 20022 and the Grammar of Payments

2.1 ISO 20022: Structured Financial Message Exchange

2.2 Terminology: Understanding Payment States

2.3 Diagram: Message Flow in Real-Time Payment

3. Event-Driven Architecture: From Kafka/Pulsar to Business Logic

3.1 Event Stream Topology

3.2 Consumer Groups and Parallelism

3.3 Diagram: Event-Driven Topology

3.4 Why Kafka/Pulsar Over Direct Service-to-Service?

4. Settlement Mechanics: Instant Gross vs. Deferred Net

4.1 Instant Gross Settlement (FedNow, SEPA Instant)

4.2 Deferred Net Settlement (Legacy ACH, Wire Clearing)

4.3 Diagram: Settlement Mechanics

4.4 Scale Numbers: Reserve and Throughput

5. Fraud Detection in Sub-Second Windows

5.1 Streaming ML: Real-Time Feature Extraction

5.2 Streaming ML Architecture

5.3 Diagram: Fraud Detection in Sub-Second Windows

5.4 Handling Post-Transaction Fraud

6. Exactly-Once Delivery and Idempotency

6.1 The Problem: Duplicates in Distributed Systems

6.2 Idempotency Architecture

6.3 Idempotency + Event Stream

6.4 Diagram: Idempotency + Exactly-Once

7. Reconciliation at Scale

7.1 The Challenge: End-of-Day Matching

7.2 Matching Algorithm

7.3 Exception Handling

7.4 Reconciliation State Machine

7.5 Diagram: Reconciliation Process

8. System Design: FedNow, UPI, and SEPA Instant

8.1 FedNow (U.S. Federal Reserve)

8.2 UPI (NPCI, India)

8.3 SEPA Instant (EBA, Europe)

8.4 Comparative Table

9. Advanced Patterns and Challenges

9.1 Handling Late-Arriving Data

9.2 Cascading Reversals

9.3 Cross-Border Real-Time Payments

9.4 Consumer Confidence and Disputes

10. Engineering Insights: Why Event-Driven Wins

10.1 Fault Isolation

10.2 Independent Scaling

10.3 Durability

10.4 Observability and Debugging

10.5 Replay and Auditing

11. Operational Considerations

11.1 Monitoring and Alerting

11.2 Disaster Recovery

11.3 Compliance and Regulations

12. Future Directions

12.1 Blockchain-Based Settlement

12.2 Machine Learning for Fraud and Risk

12.3 Cross-Border Real-Time

13. Conclusion

References and Further Reading

Updated FAQ (October 2026)

What is real-time payment infrastructure?

What is the FedNow transaction limit in October 2026?

How big is UPI in 2026 and what changes on 15 October?

What is Verification of Payee and when did it become mandatory?

How fast must SEPA Instant payments be?

Is the RTP network limit the same as FedNow?

Sources and Further Reading

Related

1 Comment

Leave a Reply Cancel reply