MQTT Shared Subscriptions: Consumer Groups, Load Balancing and Backpressure at Scale

A fleet of 200,000 meters publishes one reading every ten seconds. That is 20,000 messages per second landing on a single topic tree, and your ingestion service has to write each reading exactly once into a time-series store. One subscriber cannot keep up, and two ordinary subscribers would each receive every message and double-write everything. This is the problem MQTT shared subscriptions solve: several clients join a named group, the broker delivers each matching message to only one member, and you scale consumption by adding workers instead of rewriting the protocol.

The feature sounds trivial, and the basic syntax is. The difficulty is everything around it: what happens to an unacknowledged QoS 1 message when a worker dies, why ordering disappears, how the broker decides who is “next”, what changes when the broker is a cluster, and how a slow worker is supposed to push back. The MQTT 5 specification deliberately leaves many of those choices to the implementation.

What this covers: the exact $share/{ShareName}/{filter} semantics from the OASIS spec, dispatch strategies in EMQX, HiveMQ and VerneMQ (with what is and is not documented), redelivery rules, Receive Maximum as the real backpressure mechanism, a comparison with Kafka consumer groups, capacity-planning arithmetic, and a runnable Python worker pool.

Context and Background

Before MQTT 5, a common way to scale consumers was a hand-rolled pattern: a broker-side rule engine, a proprietary $queue/ topic prefix, or a front process that re-published to per-worker topics. Brokers such as EMQX and HiveMQ shipped their own shared-subscription features years before the standard existed. HiveMQ states on its own material that it released the feature before the MQTT 5 specification was finalised because of customer demand. MQTT 5.0 then standardised the topic-filter syntax, which is why a client written for one broker usually works unchanged on another.

The OASIS MQTT Version 5.0 specification defines the feature in section 4.8.2. The central sentence is that a publication matching a shared subscription’s filter “is only sent to one of its subscribing Sessions”, and that the server is “free to choose, on a message by message basis, which Session to use and what criteria it uses to make this selection”. Read the second half carefully. The standard guarantees exactly one recipient per group per message. It guarantees nothing about fairness, affinity or ordering. Everything in the strategy section below is therefore broker-specific behaviour, not protocol.

This matters for the wider IoT architecture. If you are moving from a purely edge-driven topology, such as the one we described in MQTT store-and-forward at the edge gateway, the shared subscription sits on the opposite side of the broker: it is how the central tier scales out. Your topic design also determines how well it works. A flat firehose topic distributes evenly but destroys per-device ordering; a structured namespace such as the one in Sparkplug B versus plain MQTT topics gives you keys to hash on.

One more piece of vocabulary. In Kafka, the equivalent construct is a consumer group; in MQTT circles people say “mqtt consumer groups” informally, though the spec calls it a Shared Subscription and the group identifier a ShareName. The analogy is useful and also dangerous, because Kafka’s group assigns partitions with stable ownership and offsets, whereas MQTT’s group assigns individual messages with no log behind them. We return to that gap in the Kafka section.

Shared Subscription Semantics: What the MQTT 5 Spec Actually Guarantees

Direct answer: A client subscribes to $share/{ShareName}/{filter}. All sessions using the same ShareName and filter form one group. For every matching publication the broker sends one copy to exactly one session in that group, plus one copy to each other group and each ordinary subscriber. The spec fixes the topology, not the selection algorithm.

Figure 1: One publication is copied once per group and once per non-shared subscriber. Inside a group the broker picks a single member per message.

Figure 1 shows the fan-out rule. Two groups subscribed to the same filter are independent: each gets its own copy, which is how you run an ingestion pool and an archiver pool against the same topic without them competing. A plain subscriber, such as a live dashboard, still receives everything.

Syntax rules you can verify

The spec makes four syntactic rules normative. The filter MUST start with $share/ and contain a ShareName at least one character long (MQTT-4.8.2-1). The ShareName MUST NOT contain /, + or #, and MUST be followed by / and then a valid topic filter (MQTT-4.8.2-2). So $share/ingest/factory/+/telemetry is valid, $share//factory/# is not, and $share/a+b/x is not. The remainder after the second slash is an ordinary filter with ordinary wildcards.

A subtle point from the spec’s examples: $share/consumer1//finance matches the same topics as a plain subscription to /finance, and two shared subscriptions with the same ShareName but different filters are distinct subscriptions with no implied relationship. A ShareName is only meaningful together with its filter. Teams that reuse one group name across several filters sometimes assume the group “balances across all of them”; it does not.

Scope: the server, not the session

The spec’s non-normative comment says shared subscriptions are “defined at the scope of the MQTT Server, rather than of a Session”. Membership works like this. A shared subscription is created by the first SUBSCRIBE using the filter. Later sessions subscribing to the same filter become additional members. A session leaves by sending UNSUBSCRIBE with the full shared filter, or by terminating. The subscription lives as long as at least one session is attached, and, importantly, “ends, and any undelivered messages associated with it are deleted, when there are no longer any Sessions subscribed to it.”

That last clause is a failure mode worth remembering. If you do a rolling restart that briefly drops every worker with a clean session, the group can vanish, and messages published in that gap are not queued for the group. HiveMQ’s own FAQ says this outcome is broker-dependent, with some brokers keeping the subscription and others removing it. The fix is operational: keep at least one member always attached, or use persistent sessions with a sensible expiry so that members are still “subscribed” while briefly offline.

Retained messages and No Local

Two details that trip up migrations. First, retained messages are not delivered when a session joins a shared subscription, whether it is the first member or a later one. If your worker pool expects to bootstrap state from retained topics, subscribe with a separate non-shared subscription for that purpose. Second, shared subscriptions interact with the subscription option No Local in a restricted way: the spec makes it a protocol error to set No Local on a shared subscription, so a worker cannot use that option to avoid receiving its own re-publishes. Design reply paths on different topics instead.

QoS is negotiated per member

Members may request different QoS levels, and the server decides what maximum QoS to grant each one. When delivering, the server MUST respect the QoS granted to the chosen member (MQTT-4.8.2-3). Consequence: a group in which one worker negotiated QoS 0 and the rest QoS 1 gets inconsistent delivery guarantees depending on which worker the broker picks. Make the group homogeneous in code review, and assert on the granted QoS in the SUBACK reason codes at startup.

A message that matches both a shared and a non-shared subscription of the same client is delivered twice: once by virtue of the plain subscription and once as a shared pick. This is spec-correct and surprises people who subscribe to # for debugging in the same client identity as a worker.

Load-Balancing Strategies and Redelivery in Real Brokers

Direct answer: Because the spec leaves selection open, each broker ships its own policies. EMQX offers random, round-robin, per-group round-robin, sticky, local, hash-by-clientid and hash-by-topic. VerneMQ offers random, prefer-local, prefer-online-before-local and local-only. HiveMQ documents round-robin-style turn taking. Pick the policy by the property you need: even load, locality or key affinity.

What each broker documents

EMQX’s shared subscription documentation lists the most options, set through mqtt.shared_subscription_strategy. As documented there: random picks a random member per message; round_robin (the default) delivers in turn, tracked per publishing client; round_robin_per_group tracks progress per node; sticky keeps sending to one member until it disconnects or its session ends; local prefers members on the node that received the message and otherwise picks at random across the cluster; hash_clientid sends all messages from one publisher client ID to the same member; hash_topic does the same for a topic name. EMQX also still accepts the deprecated $queue/ prefix, equivalent to $share/$queue/.

VerneMQ’s shared subscription page documents a shared_subscription_policy setting with four values: random, prefer_local (random on the local node, else random on a remote node), prefer_online_before_local (as prefer_local, but if the only local members are offline it looks for an online member elsewhere), and local_only. The page does not state a default, so check your version. It also notes there is no shared offline queue: each client keeps its own.

HiveMQ’s public material describes one subscriber being chosen per group for each message, “typically in round-robin order”, and says clients can join or leave at any time with the load rebalancing across the new membership. I could not retrieve HiveMQ’s detailed configuration pages in this research pass, so treat any HiveMQ strategy knobs beyond that description as unverified and check the current HiveMQ docs. Mosquitto’s open-source broker support for the feature, and any distribution policy it uses, I did not verify either; test it before relying on it. Brokers that did not appear in my research are likewise unverified.

Choosing a strategy

Round-robin and random give the best load spread when messages are independent and uniformly sized. They are the right default for stateless ingestion. The catch is that round-robin as documented by EMQX is tracked per publisher connection and resets when that publisher reconnects, so spread is statistical, not exact. With a handful of high-rate publishers and many workers you can see persistent skew, which is easy to mistake for a slow worker.

Hash strategies trade evenness for affinity. hash_clientid is attractive when each device is its own publisher: all of one device’s messages go to one worker, which restores per-device ordering while the member set is stable, and allows worker-local caches such as the last state of a digital twin. The weakness is skew. If 5 percent of devices produce 60 percent of traffic, five percent of the keys pin 60 percent of load to a few workers, and no amount of adding workers fixes it. It also remaps on membership change: when a worker joins or leaves, assignments shift (the exact remap behaviour is broker-specific and not documented in what I retrieved), so affinity is best-effort, not a contract.

Sticky is a different animal. It sends everything to one member until that member goes away. Use it for active-standby: one worker processes, the others stay warm. It is a poor load balancer by design.

Locality policies matter in clusters (Figure 3 later covers this). local and prefer_local cut inter-node hops, which saves bandwidth and latency, but they concentrate load on whichever node the publishers connect to. If publishers are skewed across nodes, workers on the busy node saturate while workers elsewhere idle.

QoS 1 and QoS 2 when a consumer dies

This is the section of the spec that deserves a careful read, because it defines what you can promise your users.

For QoS 2, the server MUST complete delivery to the chosen client when it reconnects (MQTT-4.8.2-4), and if that client’s session terminates first, the server MUST NOT send the message to any other member (MQTT-4.8.2-5). The message is bound to the chosen member. This protects the exactly-once handshake and means a QoS 2 message can be lost to the group if its member’s session is destroyed.

For QoS 1 the rules are softer. If the connection breaks before PUBACK, the server MAY wait for reconnect and retransmit. If the session terminates before reconnecting, the server SHOULD send the message to another group member, and MAY try another member as soon as it loses the connection. SHOULD and MAY are not MUST: a conformant broker could drop it. EMQX’s documentation describes the practical behaviour: when a session expires, pending QoS 1 and QoS 2 messages in its send queue and QoS 1 messages in its inflight queue are re-dispatched to other sessions in the same group, and when the last session expires pending messages are discarded.

Notice the asymmetry in EMQX’s wording: re-dispatch of QoS 2 in the send queue is described, while the spec forbids re-dispatching a QoS 2 message already in flight. The send-queue messages have not yet started the handshake, so moving them is consistent with the spec; the inflight ones are the bound ones.

Figure 2: A worker crashes holding an unacknowledged QoS 1 message. If it reconnects with the same client ID before the session expires, the broker resends with the DUP flag. If the session expires first, a conformant broker should redispatch to another member.

Two consequences follow. First, at-least-once with duplicates is the best a QoS 1 group gives you: the crash may have happened after your database write and before the PUBACK, so the same message can be processed twice, possibly by different workers. Your sink must be idempotent, for example by upserting on (device_id, timestamp) or a message-level key you embed in the payload. Second, the session expiry interval is a design parameter, not a default. Too long and a dead worker’s messages sit idle in its queue; too short and a brief network flap triggers redispatch and duplicates.

Ordering is gone, and that is a feature

Spread a topic across N workers and two consecutive messages from one device can be processed concurrently on different machines, finishing in either order. MQTT’s ordering guarantee applies per topic and subscription for a single session; a shared group splits one logical stream across several sessions, so it no longer applies. Even with hash strategies, the guarantee evaporates on membership changes, and on QoS 1 redelivery a retransmitted older message can arrive after a newer one.

Three ways to cope. Make processing commutative, for instance last-write-wins keyed on an embedded device timestamp or sequence number. Use a hash strategy to reduce reordering and accept residual risk. Or stop pretending: if strict per-device order is a requirement, shared subscriptions are the wrong primitive, and the Kafka section below explains what to use instead.

Backpressure: Receive Maximum, Inflight Windows and the Queue Behind Them

Direct answer: MQTT 5 backpressure is credit-based. Each client advertises a Receive Maximum in CONNECT; the broker may have at most that many unacknowledged QoS 1 and QoS 2 PUBLISH packets outstanding to it. When the window is full, the broker stops sending QoS above 0 to that member and messages queue. Slow workers get less, but the queue behind them is still bounded only by broker config.

The spec’s flow control section (4.9) defines a send quota. The sender initialises it to a non-zero value not exceeding the Receive Maximum (MQTT-4.9.0-1), decrements it for every QoS 1 or QoS 2 PUBLISH, and MUST NOT send further QoS above 0 PUBLISH packets when it reaches zero (MQTT-4.9.0-2). PUBACK or PUBCOMP restores one unit, as does a PUBREC carrying an error reason code. QoS 0 may continue. The default Receive Maximum when absent is 65,535, and a value of zero is a protocol error. Neither the quota nor the Receive Maximum is part of session state: both are re-initialised for each new network connection.

Why this is better than a plain prefetch

In MQTT 3.1.1 there was no standard window; brokers invented inflight limits. In MQTT 5 a consumer states its limit, and a shared group inherits useful behaviour: a worker that processes slowly holds its window full of unacknowledged messages and the broker has less reason to send it more. Whether a given broker skips full-window members when picking the next member is an implementation question. Not every broker does, and a strategy such as hash or sticky may deliberately keep choosing the full member, in which case messages pile up in that member’s queue while others idle. I did not find a vendor statement guaranteeing window-aware selection for each strategy, so verify empirically.

The real design lever is Receive Maximum, chosen to match how your worker processes. If a worker handles messages one at a time and acknowledges after the database commit, a Receive Maximum of 1 gives strict one-at-a-time delivery and the fairest balancing across workers, at the price of a broker round trip per message. A value of 100 hides network latency but means a crashed worker strands up to 100 messages that will be redelivered. A rule of thumb with illustrative numbers: choose a window of roughly (target throughput per worker) multiplied by (round-trip time plus processing time). At 500 messages per second and 40 ms combined, that is about 20 in flight.

Acknowledge after the work, not on arrival

Most client libraries acknowledge automatically when the message callback returns, which makes the callback the unit of work. If the callback hands the message to a thread pool and returns, the client has acknowledged a message that has not been processed. The window then never fills, backpressure disappears, and a crash silently loses everything queued in memory. The fix is manual acknowledgement: paho-mqtt v2 exposes manual_ack_set(True) and client.ack(mid, qos) for this reason. We use it in the sample below.

Where the pressure goes next

Once windows are full, the broker queues. Per-session queues are finite in every broker; when full, some policy discards messages (oldest or newest, QoS-dependent, and broker-specific). EMQX’s page warns that a persistent session receiving shared-topic messages while offline can overflow its queue under a long outage at a high publish rate, and recommends clean sessions with a short non-zero session expiry for v5 clients. Backpressure therefore ends at a policy decision you must make explicitly: bounded queue and drop, or unbounded and run out of memory. Monitor queue depth and dropped counters and alert before they matter.

Backpressure also does not reach the publisher. Devices keep publishing at the rate their firmware dictates, and QoS 0 messages beyond the window may be dropped silently. The protocol offers publishers only a similar window toward the broker (the broker’s own Receive Maximum in CONNACK), not a signal about downstream group health. If you need producers to slow, you need an application-level mechanism, such as a rate-limit command topic.

Broker Clustering, Kafka Comparison and a Runnable Worker Pool

What clustering changes

In a single-node broker, selecting a group member is a local lookup. In a cluster, the group’s members are spread across nodes, and the node that receives a publication must know who the members are and where they live. Cluster-wide membership state is replicated, and the chosen message is forwarded across the inter-node link if the member is remote. That makes shared subscriptions a source of east-west traffic, which is why the locality strategies exist.

Figure 3: Routing a shared-subscription message in a cluster. Locality policies avoid inter-node hops, random policies spread load across nodes, and hash policies pin a key to a worker. All end at a per-worker inflight window.

Consider the arithmetic with illustrative numbers. Suppose 20,000 messages per second at 300 bytes each, and a three-node cluster with publishers evenly distributed. With a random cross-cluster policy, a given message lands on the receiving node’s own worker one-third of the time, so about two-thirds of traffic, roughly 13,300 messages per second or 4 MB/s of payload before protocol and replication overhead, crosses the inter-node network. With a local policy, that falls toward zero, but only if workers are placed proportionally on each node. If node 1 receives 50 percent of the publishers and hosts one third of the workers, those workers carry 50 percent of the load. The speed-up of locality is real; so is the hot-spot risk.

Operational guidance that follows. Place workers evenly across nodes and across the same zones as publishers. Prefer locality policies only when publisher distribution is roughly uniform, which a load balancer in front of the cluster usually gives you. Measure per-worker rates, not just group totals; a flat total can hide one saturated worker and four idle ones. HiveMQ’s public material states shared subscriptions work with load balancers in clustered set-ups, and notes that improving subscriber node locality reduces intra-cluster traffic, which matches this reasoning.

Cluster membership changes add another failure mode. When a node leaves, its sessions’ state is lost or migrated depending on broker and session settings, and workers connected to it reconnect elsewhere, producing a burst of redeliveries. If you are on EMQX and planning a major-version move, the shared-subscription strategy settings are one of the items to retest; our EMQX 6.3 LTS versus 5.8 LTS migration notes cover the broader upgrade checklist. Always re-verify the option names against the version you run, since my verified source here is the current-latest documentation page.

Shared subscriptions versus Kafka consumer groups

Teams reach for the Kafka analogy often, so be precise about where it holds.

Property	MQTT shared subscription	Kafka consumer group
Unit of assignment	Each message, chosen by broker policy	Each partition, owned by one consumer
Ordering	None across the group; best-effort with hash policies	Strict per partition
Durable log	No; messages are queued per session until acked or expired	Yes; retained by time or size
Replay	No	Yes, by resetting offsets
Consumer position	Broker tracks inflight per session	Committed offsets per group
Max parallelism	Number of members, no hard cap	Number of partitions
Rebalance	Immediate, membership change affects the next pick	Coordinated rebalance, partition reassignment
Slow consumer	Window fills, queue grows or drops	Lag grows; log retains the data
Fit	Device telemetry fan-in at the edge of the platform	Durable event backbone

Two properties dominate. Kafka decouples “when the data arrived” from “when it is processed” with a log, so a stalled consumer group loses nothing until retention runs out. An MQTT shared group has no log: a stalled group loses data when its queues fill. And Kafka orders by partition key, so per-device order can be guaranteed by keying on device ID. Neither guarantee exists in a shared group.

Figure 4: Two common patterns. Stateless workers consume directly from a shared subscription, or a bridge writes into Kafka so that several consumer groups can replay and process independently.

The decision follows from what you need after the broker. If the consumer is a stateless writer into a store that tolerates duplicates, shared subscriptions are simpler and cheaper: no extra system. If you need replay, multiple independent downstream consumers, strict per-key ordering, or processing that can lag by hours, bridge into Kafka (through a broker-side connector, a Kafka Connect source, or a small bridge process) and let Kafka consumer groups own scale-out. A bridge process itself is typically the sole member or a small shared group on the MQTT side, with its throughput and acknowledgement behaviour becoming the thing to capacity-plan. The Kafka-side facts here (partition-level assignment, offsets, replay) come from general knowledge of Kafka’s design; consult the Apache Kafka documentation for your version before committing.

A runnable worker pool with paho-mqtt v2

The sample below uses paho-mqtt 2.x, which requires you to pass a callback API version. It connects with MQTT 5, a non-zero session expiry, an explicit Receive Maximum, manual acknowledgement, and a bounded worker thread pool. Acknowledgement happens only after the work function returns successfully. I have not run this against a live broker in this writing pass, so treat it as a reviewed template and test it against your own broker; the paho API names (CallbackAPIVersion.VERSION2, manual_ack_set, ack) are from the paho-mqtt 2.x documentation as I know it.

import os, queue, threading, json
import paho.mqtt.client as mqtt
from paho.mqtt.packettypes import PacketTypes
from paho.mqtt.properties import Properties
from paho.mqtt.subscribeoptions import SubscribeOptions

HOST = os.getenv("MQTT_HOST", "localhost")
GROUP = "ingest"
FILTER = "factory/+/telemetry"
SHARED = f"$share/{GROUP}/{FILTER}"
WORKERS = 4
RECEIVE_MAX = 32          # broker never has >32 unacked QoS1 to us
SESSION_EXPIRY = 60       # seconds; dead worker's msgs redispatch after this

work_q = queue.Queue(maxsize=RECEIVE_MAX)   # bounded: mirrors the window
client = mqtt.Client(
    mqtt.CallbackAPIVersion.VERSION2,
    client_id=os.getenv("WORKER_ID", f"worker-{os.getpid()}"),
    protocol=mqtt.MQTTv5,
)
client.manual_ack_set(True)    # we ack after the work, not on arrival

def process(payload: bytes) -> None:
    doc = json.loads(payload)
    # idempotent write: upsert on (device, ts), safe under redelivery
    print("stored", doc.get("device"), doc.get("ts"))

def worker():
    while True:
        msg = work_q.get()
        try:
            process(msg.payload)
            client.ack(msg.mid, msg.qos)       # PUBACK after success
        except Exception as exc:
            # no ack: broker keeps it inflight and redelivers
            print("failed, leaving unacked:", exc)
        finally:
            work_q.task_done()

def on_connect(c, userdata, flags, reason_code, properties):
    if reason_code.is_failure:
        print("connect failed:", reason_code); return
    rc, mid = c.subscribe(SHARED, options=SubscribeOptions(qos=1))
    print("subscribed", SHARED, rc)

def on_message(c, userdata, msg):
    work_q.put(msg)   # blocks when full -> stops acking -> window fills

client.on_connect = on_connect
client.on_message = on_message

props = Properties(PacketTypes.CONNECT)
props.SessionExpiryInterval = SESSION_EXPIRY
props.ReceiveMaximum = RECEIVE_MAX

for _ in range(WORKERS):
    threading.Thread(target=worker, daemon=True).start()

client.connect(HOST, 1883, keepalive=30,
               clean_start=mqtt.MQTT_CLEAN_START_FIRST_ONLY,
               properties=props)
client.loop_forever()

Several design choices here are deliberate. The queue size equals the Receive Maximum so the process never holds more than the broker allows. A failed message is not acknowledged, so it stays inflight and is redelivered when the connection cycles. The client ID is stable across restarts (set WORKER_ID per replica) so a restarted worker can resume its session within the expiry window. Blocking put inside the network callback does stall the paho network loop under saturation; in production you may prefer to run the loop in a thread and apply a pause or a larger buffer, but never an unbounded one. Also note that an unacknowledged failing message will be retried forever; add a retry counter and a dead-letter topic for poison messages, publishing the bad payload to something like dlq/ingest before acknowledging.

To scale out, run more replicas with different client IDs and the same SHARED filter. Nothing else changes, which is the entire appeal. To test redelivery, kill a replica with kill -9 mid-processing and watch the broker resend after reconnect or redispatch after the session expiry.

Capacity Planning with Worked Numbers

All figures here are illustrative; substitute your own measurements.

Start from demand. 200,000 devices at one message per 10 seconds is 20,000 messages per second average. Telemetry is rarely flat, so apply a peak factor, say 3x during a reconnect storm or a shift change, giving a design target of 60,000 messages per second.

Next, per-worker service time. If a worker writes a batch to a database and each message costs 2 ms of CPU and I/O on one thread, one thread handles 500 messages per second. A four-thread worker handles about 2,000, ignoring contention. To sustain 60,000 you need 30 such workers, and 40 with a 25 percent headroom margin. Average load needs only 10, so autoscaling from 10 to 40 is plausible, and shared subscriptions handle join and leave without reconfiguration.

Then the window. At 2,000 messages per second per worker and a 10 ms broker round trip with 2 ms average processing, in-flight demand is about 2,000 times 0.012, which is 24 messages. A Receive Maximum of 32 to 64 covers it with slack. Across 40 workers, the maximum messages exposed to loss-and-duplicate on a simultaneous failure is 40 times 64, or 2,560, which is a clean bound on your duplicate burst after a mass restart.

Last, memory and queues. A 300-byte payload plus roughly 100 to 200 bytes of broker overhead per queued message (a rough estimate, not a measurement) means a million queued messages cost on the order of 400 to 500 MB across the cluster. At a 60,000 per second peak, a one-minute worker outage accumulates 3.6 million messages if the group is fully stalled, so queue limits and session expiry decide whether you survive that minute. Set limits explicitly and test the failure mode on a staging cluster.

Trade-offs, Gotchas, and What Goes Wrong

Duplicate processing is the default, not an edge case. QoS 1 redelivery after a crash, a session expiry redispatch, or a network flap all create it. If your sink is not idempotent, shared subscriptions will corrupt data quietly. Idempotency keys, upserts and sequence numbers are the cure.

Ordering silently breaks. Engineers test with one worker and see ordered output, then scale to ten and meet reordering in production. Test concurrency early with at least three workers and a deliberately slow one.

Group disappearance on full restart. The spec says the shared subscription ends and undelivered messages are deleted once no sessions remain. A blue-green deployment that stops all workers before starting new ones can drop a window of traffic. Overlap old and new workers, or use persistent sessions with a sufficient expiry.

Client ID collisions. Two workers with the same client ID kick each other off in a loop, and each reconnect triggers redelivery. Derive IDs from pod names or hostnames, never from a constant.

Mixed QoS and mixed subscriptions. A worker that also holds a plain subscription to the same topic receives duplicates, as the spec describes. Retained messages never arrive through the shared path.

Poison messages. A payload that always throws and is never acknowledged will be retried indefinitely, potentially pinning a window slot. Cap retries and dead-letter.

Hash-strategy skew and remapping. As discussed, a few hot keys defeat balancing, and membership change remaps keys.

Cluster blind spots. Dashboards that show only group totals hide one overloaded worker. Per-member inflight, queue length and ack latency are the metrics that matter. Broker metrics names differ by product, so I will not guess them here.

Finally, remember what shared subscriptions are not: not a durable queue, not a replay mechanism, not an ordering primitive. When the requirement list contains any of those words, reach for Kafka or a broker-native queue feature and keep MQTT as the device-facing transport.

Practical Recommendations

Start with the simplest design that satisfies your requirements. For stateless, idempotent ingestion, use a shared subscription with the broker’s default balanced strategy, QoS 1, manual acknowledgement after the durable write, a Receive Maximum sized from your latency and throughput arithmetic, and a short non-zero session expiry. Scale workers horizontally and watch per-member metrics.

If you need per-device ordering within the group, try a hash-by-publisher strategy, embed sequence numbers and make processing tolerant of residual reordering. If you need hard ordering, replay or several independent consumers, bridge to Kafka and put your scale-out there.

Treat broker-specific behaviour as something to test, not assume. The standard promises one recipient per group per message and little more.

Checklist:

[ ] Sink is idempotent (upsert key or dedupe table).
[ ] Workers acknowledge after the durable write, using manual ack.
[ ] Receive Maximum is set deliberately from throughput times latency.
[ ] Session expiry is short but non-zero; client IDs are stable per replica and unique across replicas.
[ ] At least one worker is always attached during deployments.
[ ] Strategy chosen for a reason (spread, locality, affinity) and re-verified per broker version.
[ ] Queue limits, drop policy and per-member lag alerts are configured.
[ ] A dead-letter path exists for poison messages.
[ ] Failure drill done: kill a worker mid-batch and a whole node, then check duplicates and loss.

Frequently Asked Questions

What is an MQTT shared subscription?

It is a subscription whose topic filter starts with $share/{ShareName}/. Every session that subscribes with the same ShareName and filter joins one group, and the broker sends each matching message to only one member of that group. It lets you scale consumers horizontally without writing broker plugins, and MQTT 5.0 standardised the syntax in section 4.8.2 of the specification.

How is a shared subscription different from a normal one?

A normal subscription delivers a copy to every subscribed client. A shared subscription delivers one copy per group, to a single member, and separate groups each get their own copy. Retained messages are not sent when a member joins, and No Local cannot be set on it. Selection of the member is up to the broker.

What happens to unacknowledged QoS 1 messages if a worker crashes?

The spec lets the broker wait for the worker to reconnect and retransmit, and says it SHOULD redeliver to another member if the session terminates. Brokers such as EMQX document redispatch of inflight QoS 1 messages when a session expires. Expect duplicates, so make processing idempotent. QoS 2 messages are the exception: the spec forbids sending them to another member.

Do MQTT shared subscriptions preserve message order?

No, not across the group. The messages of one topic are spread across several sessions, which process them concurrently. Hash strategies such as EMQX’s hash_clientid or hash_topic reduce reordering by sending a key to the same member, but membership changes and redelivery can still reorder. For strict ordering use a Kafka partition key.

How does backpressure work with shared subscriptions?

Each member advertises a Receive Maximum in CONNECT, and the broker may not have more unacknowledged QoS 1 or 2 messages outstanding to it than that. When a member is slow its window fills and the broker queues or drops further messages. The publishers are not told, so protect the pipeline with queue limits and monitoring.

Should I use shared subscriptions or Kafka consumer groups?

Use shared subscriptions when consumers are stateless, writes are idempotent and you want no extra infrastructure. Use Kafka when you need durable storage, replay, multiple independent consumer groups or per-key ordering. Many IoT platforms combine them: MQTT at the edge and Kafka behind a bridge for the durable backbone.

MQTT 5 Shared Subscriptions: Consumer Groups, Load Balancing and Backpressure at Scale (2026)