Introduction: From IBM Labs to One Billion Devices
In 1999, Andy Stanford-Clark and Arlen Nipper at IBM created a lightweight messaging protocol to monitor an oil pipeline across the desert. That protocol—MQTT (Message Queuing Telemetry Transport)—has since become the de facto standard for IoT messaging, powering over 1 billion connected devices globally. Today, MQTT runs at the heart of AWS IoT Core, Microsoft Azure IoT Hub, HiveMQ’s cloud platform, and countless enterprise deployments from smart cities to industrial manufacturing floors.
Why has MQTT survived and thrived for 25+ years? Because it solves a fundamental problem: how do you move data reliably across unreliable networks with minimal overhead and battery drain?
This guide cuts through vendor marketing and teaches you MQTT from first principles—from the raw packet structure to clustering strategies, from QoS semantics to failure modes. By the end, you’ll understand not just how MQTT works, but when to use it, when not to, and how to architect systems that scale.
TL;DR
What is MQTT? A lightweight publish-subscribe protocol designed for bandwidth-constrained, high-latency, unreliable networks. Clients publish messages to topics on a central broker; subscribers receive only messages from topics they’ve subscribed to.
Why it matters: Requires minimal CPU, memory, and network bandwidth compared to HTTP polling or persistent REST connections. Handles network failures gracefully via reconnection logic and persistent sessions. Supports three quality-of-service levels (QoS 0, 1, 2) to match application reliability needs.
Current landscape: MQTT 3.1.1 dominates production (RFC 3.1.1); MQTT 5.0 (2017) adds enhanced features (user properties, shared subscriptions, reason codes, session expiry). MQTT-SN extends the protocol to ultra-constrained devices. Sparkplug B (2022+) wraps MQTT for industrial automation with strict sequencing and payload format rules.
Adoption: AWS, Azure, Google Cloud, HiveMQ, Mosquitto (open-source), Emqx, VerneMQ, and hundreds of embedded broker implementations.
Table of Contents
- Key Concepts
- Why MQTT Exists
- MQTT Packet Structure
- QoS 0/1/2 and Message Flows
- MQTT 5.0 Features Beyond 3.1.1
- Broker Architecture for Scale
- MQTT 3.1.1 vs MQTT 5.0 vs MQTT-SN
- Edge Cases & Failure Modes
- Implementation Guide
- FAQ
- Where MQTT Is Heading
- References
- Related Posts
Key Concepts
Before diving into packets and QoS, understand these core terms:
Client
Any device or application that connects to an MQTT broker. Clients can be publishers (send messages), subscribers (receive messages), or both. A temperature sensor, a mobile app, a cloud microservice—all are MQTT clients.
Broker
The central hub that accepts connections from all clients and routes messages between publishers and subscribers. The broker doesn’t process or store application logic; it only moves bytes. Examples: Mosquitto, HiveMQ, AWS IoT Core, Azure IoT Hub, EMQX.
Topic
A UTF-8 string that acts as a publish-subscribe address. Topics are hierarchical, separated by forward slashes: sensors/factory-a/line-2/temperature. Subscribers filter on topics using wildcards: sensors/+/+/temperature (matches any factory/line), sensors/# (matches anything under sensors/).
Publish & Subscribe
- Publish: Client sends
PUBLISHpacket with a payload (the message bytes) and a topic. The broker immediately delivers to all subscribed clients. - Subscribe: Client sends
SUBSCRIBEpacket listing one or more topic filters. Client receives all futurePUBLISHpackets matching those filters.
Retained Message
A message marked with the retain flag stays on the broker even after all subscribers disconnect. When a new subscriber connects, the broker delivers the last retained message on that topic. Useful for stateful data (current temperature, device status) that latecomers need to see immediately.
Session
MQTT connections are stateful. A session encompasses:
– The client’s subscriptions.
– Undelivered messages (for QoS 1 & 2) queued for the client.
– The client’s last known state.
If a client disconnects and reconnects with the same clientId and cleanSession=false, the broker resumes the old session.
Will Message
A PUBLISH packet that the broker automatically sends to a configured topic if the client disconnects abnormally (network failure, no graceful DISCONNECT). Useful for “device offline” alerts.
QoS (Quality of Service)
Three delivery guarantees:
– QoS 0 (At-Most-Once): No acknowledgment. Fire and forget. Message may be lost if broker or network fails.
– QoS 1 (At-Least-Once): Broker sends PUBACK acknowledgment. Client may resend the message repeatedly until it gets the ack, so the broker may deliver duplicates.
– QoS 2 (Exactly-Once): Four-way handshake (PUBLISH → PUBREC → PUBREL → PUBCOMP). Guarantees no duplicates even under retransmission. Slowest and most overhead.
Packet
A discrete unit of data sent over the TCP connection. Every MQTT interaction (CONNECT, PUBLISH, SUBSCRIBE, etc.) is a packet with a fixed header, optional variable header, and optional payload.
Why MQTT Exists
To understand MQTT, contrast it with HTTP, which dominated web development:
HTTP: Stateless Pull Model
Client: "Hey, is there new data?" → GET /api/data
Server: 200 OK, [data or empty]
Client: (waits, then polls again)
Client: "Is there new data yet?" → GET /api/data
Server: 200 OK, [data or empty]
Overhead per poll:
– TCP handshake (3-way) if connection reused across multiple requests.
– HTTP headers (200–500 bytes) for every request.
– Server must store no state; every request is independent.
For a sensor sending one temperature reading every 10 seconds:
– 8,640 requests/day.
– 8,640 HTTP header transactions.
– Massive battery drain on IoT devices.
MQTT: Stateful Push Model
Pub Client: PUBLISH("sensors/temp", 23.5, QoS=1) → Broker
Broker: PUBACK
Broker: PUBLISH("sensors/temp", 23.5) → Sub Client 1
Sub 1: PUBACK (QoS=1)
Broker: PUBLISH("sensors/temp", 23.5) → Sub Client 2
Sub 2: PUBACK (QoS=1)
Advantages:
– One TCP connection, many messages. After CONNECT, the client stays connected and sends/receives multiple PUBLISHes without re-establishing the connection.
– Tiny packet size. Fixed header is just 2 bytes (plus variable-length remaining-length encoding). No HTTP headers per message.
– Push, not pull. Subscribers don’t poll. The broker pushes updates immediately.
– Battery-efficient. Minimal CPU wake-ups, smaller payloads, reused TCP connection.
– Handles intermittent connectivity. Keep-alive timers detect dead connections; persistent sessions survive disconnects.
The chart above shows HTTP’s polling overhead (each dot = one request) versus MQTT’s efficient push model (continuous connection, messages only when data changes).
MQTT Packet Structure (Fixed + Variable + Payload)
Every MQTT packet follows a three-part structure:
Fixed Header (2 bytes minimum)
Byte 1:
[7:4] Message Type (1=CONNECT, 3=PUBLISH, 8=SUBSCRIBE, etc.)
[3] Duplicate flag (set if broker is resending)
[2:1] QoS level (0, 1, or 2)
[0] Retain flag
Bytes 2–5:
Remaining Length (variable-length integer encoding)
The length of everything after the fixed header
Uses continuation bits to support messages up to 256 MB
Example: 0x30 0x0a means:
– 0x30: Message type PUBLISH (0x3), DUP=0, QoS=0, Retain=0
– 0x0a: Remaining length = 10 bytes
Variable Header (Protocol-Dependent)
For PUBLISH packets:
– Topic name (2-byte length prefix + UTF-8 string)
– Packet ID (2 bytes, if QoS > 0)
For CONNECT packets:
– Protocol name (“MQTT”, 4 bytes after length prefix)
– Protocol version (1 byte: 4 for 3.1.1, 5 for 5.0)
– Connect flags (1 byte, encoding username/password/will/clean-session/keep-alive settings)
– Keep alive interval (2 bytes, seconds)
For SUBSCRIBE packets:
– Packet ID (2 bytes)
– List of (topic filter + QoS) pairs
Payload
The actual message data:
– For PUBLISH: the sensor reading, status, or command bytes.
– For CONNECT: client ID, username, password, will topic/message.
– For SUBSCRIBE: topic filters and their requested QoS levels.
– For DISCONNECT: empty (in MQTT 3.1.1) or reason code (in MQTT 5.0).
Why this structure? The fixed header is tiny (2 bytes) and synchronization is trivial (read byte 1, decode remaining length, read payload). Variable-length integers keep small messages small; large messages add only a few extra length bytes. No bloated headers like HTTP.
QoS 0/1/2 and Message Flows
MQTT’s three quality-of-service levels are the protocol’s backbone. Each offers different reliability guarantees and overhead.
QoS 0: At-Most-Once (Fire and Forget)
Publisher Broker Subscriber
| | |
+------ PUBLISH ------->| |
| +------ PUBLISH ------->|
| | |
Behavior:
– Publisher sends PUBLISH; broker immediately forwards to subscribers.
– No acknowledgment. Publisher has no way to know if the message reached the broker or subscribers.
– If the broker crashes mid-broadcast, or the network fails, the message is lost.
– Most efficient. Minimal overhead: one packet from publisher, one per subscriber.
Use case: Non-critical telemetry (ambient temperature every 10 seconds where individual readings aren’t critical). Losing one reading is acceptable.
Latency: Lowest (no ACKs).
QoS 1: At-Least-Once
Publisher Broker Subscriber
| | |
+-- PUBLISH (id=1) ---->| |
| +-- PUBLISH (id=1) ---->|
|<----- PUBACK ------+ | |
| | |<----- PUBACK --------+
| | |
| (store in queue until PUBACK)
Behavior:
– Publisher sends PUBLISH with a unique packet ID (1–65535).
– Broker forwards PUBLISH to subscribers and awaits their PUBACK.
– Once broker gets PUBACK from all subscribers (or timeout), it sends PUBACK to publisher.
– If publisher doesn’t get PUBACK within a timeout (typically a few seconds), it resends the PUBLISH with the DUP flag set.
– Duplicates possible. If the network fails after publisher receives PUBACK but before it marks the message as sent, the publisher may resend, and the broker may deliver it twice.
– Broker queues unacknowledged messages and resends them to subscribers that are offline (when they reconnect).
Use case: Important alerts (temperature threshold breach, device error) where missing data is costly, but occasional duplicates are tolerable.
Latency: Medium (one RTT for ACK).
QoS 2: Exactly-Once
Publisher Broker Subscriber
| | |
+--- PUBLISH (id=1) --->| |
| +--- PUBLISH (id=1) --->|
|<----- PUBREC -------+ | |
| | |<----- PUBREC --------+
+--- PUBREL (id=1) --|-> | |
| +--- PUBREL (id=1) --->|
|<----- PUBCOMP ------+ | |
| | |<----- PUBCOMP ------+
| |
Behavior:
– Four-way handshake ensures no duplicates.
– Publisher sends PUBLISH; broker responds with PUBREC (publish received).
– Publisher sends PUBREL (publish release); broker responds with PUBCOMP (publish complete).
– Broker guarantees it will deliver the message exactly once to each subscriber, never twice.
– Both broker and publisher persist packet state during the handshake; if either crashes, they recover from their stored state and resume.
Use case: Financial transactions, critical control commands (pump shutdown, valve open), billing events. The overhead is worth certainty.
Latency: Highest (two RTTs).
QoS Negotiation
When a subscriber subscribes to a topic, it specifies the maximum QoS it accepts:
SUBSCRIBE
Topic: "alerts/critical" QoS: 1
Topic: "telemetry/temp" QoS: 0
The broker delivers at the minimum of:
– The publisher’s PUBLISH QoS, and
– The subscriber’s requested QoS.
Example: Publisher sends at QoS 2; subscriber asks for QoS 0 → broker delivers at QoS 0 (no ACK required).
MQTT 5.0 Features Beyond 3.1.1
MQTT 5.0 (ratified 2017) maintains backward compatibility with 3.1.1 but adds powerful features for modern systems.
Key Additions
User Properties
Attach arbitrary key-value metadata to PUBLISH, CONNECT, SUBSCRIBE, and DISCONNECT packets. Enables application-level tagging without changing the core protocol.
PUBLISH("sensors/temp", payload=23.5)
User Property: "location" = "warehouse-a"
User Property: "timestamp" = "2026-04-18T10:30:45Z"
Applications parse user properties to route, log, or enrich messages without broker involvement.
Shared Subscriptions
In MQTT 3.1.1, if two subscribers with the same topic filter connect, both receive every message. For load-balancing a high-volume topic across multiple workers, you’d need external logic.
MQTT 5.0 introduces shared subscriptions:
SUBSCRIBE "$share/worker-pool/orders"
Only one member of the worker-pool group receives each message, distributing load automatically.
Reason Codes
MQTT 3.1.1 uses simple 0/non-zero responses. MQTT 5.0 adds detailed reason codes:
0x00: Success0x04: Disconnect with Will Message (client initiated disconnect but kept will on broker)0x80: Unspecified error0x81: Malformed Packet0x82: Protocol Error0x83: Implementation-specific error
Applications can retry intelligently based on the reason.
Request/Response Pattern
MQTT traditionally supports only pub-sub. MQTT 5.0 adds RPC-like request/response:
Client A: PUBLISH("$req/cmd/status", "get-cpu-usage")
Response Topic: "$resp/cmd/status-reply/unique-id"
Client B: (subscribes to $req/...)
Client B: PUBLISH("$resp/cmd/status-reply/unique-id", "cpu: 45%")
Correlation Data: unique-id
The Response Topic field tells the responder where to send its reply. The Correlation Data lets the requester match requests to responses.
Session Expiry
MQTT 3.1.1 has no standard for how long the broker keeps a session after disconnect. MQTT 5.0 standardizes:
CONNECT
Session Expiry Interval: 3600 // seconds
If the client doesn’t reconnect within 1 hour, the broker purges the session and undelivered messages. Lets operators set predictable cleanup windows.
Topic Alias
Large topic names bloat every PUBLISH. Topic alias maps a 2-byte integer to a topic:
PUBLISH("house/ground-floor/living-room/thermostat/current-temperature", payload=21.5)
Topic Alias: 5
// Subsequent publishes on same topic can omit the name:
PUBLISH(Topic Alias: 5, payload=21.6)
Saves bandwidth on high-frequency topics.
Enhanced Authentication
MQTT 3.1.1 supports only username/password in the CONNECT packet. MQTT 5.0 introduces AUTH packets for challenge-response flows (e.g., SCRAM, OAuth2-style flows).
Broker Architecture for Scale
A single Mosquitto instance can handle thousands of clients. But at 100K+ clients or when you need zero downtime, you need clustering and sharding strategies.
Clustering
Multiple broker nodes share subscriptions and messages via replication:
Broker-A Broker-B Broker-C
| | |
| <--- Topic Replication ----> | <--- Topic Replication ----> |
| <--- Message Replication ---> |
|
+-- Client-1 +-- Client-2 +-- Client-3
+-- Client-2 +-- Client-3
When Client-1 (connected to Broker-A) publishes to sensors/temp, Broker-A replicates the message to Brokers B and C, which deliver to their respective subscribers.
Clustering benefits:
– High availability. If one broker fails, clients on others survive.
– Load distribution. Clients spread across nodes.
– Geographic distribution. Brokers in different regions, with local failover.
Tools: HiveMQ, EMQX, VerneMQ support native clustering. AWS IoT Core and Azure IoT Hub offer managed clustering.
Sharding
For extreme scale (millions of clients), partition clients by topic or clientId across brokers:
Broker A: Handles topics "house/*"
Broker B: Handles topics "factory/*"
Broker C: Handles topics "vehicle/*"
(Router / load balancer redirects CONNECT to the right broker)
Tradeoff: Clients can only publish/subscribe to topics on their assigned broker. Cross-shard topics require bridges.
MQTT Bridges & Gateways
A broker-to-broker bridge relays messages between clusters or between MQTT and other protocols:
Edge Broker (Mosquitto) <--- Bridge MQTT ---> Cloud Broker (HiveMQ)
|
+-- Industrial Devices (MQTT-SN Gateway)
+-- Sparkplug B Converters
MQTT-SN (Sensor Networks) compresses MQTT for ultra-constrained devices (6LoWPAN, Zigbee, LoRaWAN gateways). The gateway translates between MQTT-SN and standard MQTT.
Sparkplug B is a Payload Format Specification that wraps MQTT for industrial automation, adding strict sequencing and node versioning.
MQTT 3.1.1 vs MQTT 5.0 vs MQTT-SN
| Feature | MQTT 3.1.1 | MQTT 5.0 | MQTT-SN |
|---|---|---|---|
| Year Adopted | 2014 | 2017 | 2013 |
| TCP/IP Required | Yes | Yes | No (UDP, Zigbee, etc.) |
| Packet Size (hello) | ~20 bytes | ~30 bytes | ~3 bytes |
| User Properties | ✗ | ✓ | ✗ |
| Shared Subscriptions | ✗ | ✓ | ✗ |
| Reason Codes | ✗ (0 or error) | ✓ (detailed) | ✗ |
| Request/Response | ✗ | ✓ | ✗ |
| Session Expiry | Broker-defined | Client-defined | N/A |
| Topic Alias | ✗ | ✓ | ✓ (topic IDs) |
| Auth Challenge | ✗ | ✓ | ✗ |
| Maturity | High | High | Stable, niche |
| Use Case | General IoT, cloud | Modern IoT, enterprises | Constrained devices |
MQTT 3.1.1 dominates production environments due to ubiquitous library support and proven reliability. MQTT 5.0 is rising for new deployments, especially where shared subscriptions or request/response patterns save complexity. MQTT-SN is standard in industrial wireless (6LoWPAN, Zigbee) but less common in IP networks.
Edge Cases & Failure Modes
MQTT’s simplicity hides subtle failure modes. Understanding them prevents production surprises.
Retained Message Pollution
A publisher sets retain=true on every PUBLISH:
PUBLISH("device/123/status", "ONLINE", retain=true) // Broker stores
// Device crashes
PUBLISH("device/123/status", "OFFLINE", retain=true) // Broker stores
// One week later, a new subscriber connects:
SUBSCRIBE("device/#")
// Receives "OFFLINE" immediately, even though the device might be back online
Mitigation: Use retain sparingly; ensure publishers always update retained messages. Set session expiry to clean up stale state.
Will Message Misuse
A client connects with a will message:
CONNECT(clientId="sensor-1")
Will Topic: "alerts/down"
Will Message: "sensor-1 offline"
The broker stores the will. If the client crashes, the broker publishes it. But:
– Duplicate wills. If the network is flaky and the client reconnects multiple times, multiple wills may fire.
– Stale wills. If the client restarts and reconnects within seconds, the broker may send a stale “offline” message after the client is already back online.
Mitigation: Use will messages for alerts only; don’t rely on them for state. Pair with heartbeat logic.
Wildcard Storms
A client subscribes too broadly:
SUBSCRIBE("#") // Every message on the broker
If the broker has 10K topics and 100K msg/sec, this one subscriber gets hammered. If its connection is slow or its queue overflows, the broker may:
– Block other clients (back-pressure).
– Drop messages for this subscriber.
– Crash.
Mitigation: Implement per-client quotas; reject subscriptions with too many wildcards; monitor subscriber throughput.
Persistent Session Bloat
With cleanSession=false, the broker stores:
– Client subscriptions.
– Undelivered QoS 1/2 messages.
If millions of clients connect once then never reconnect, the broker’s memory grows unbounded:
1M clients × (100 bytes subscriptions + 10 KB undelivered) = 10 GB+ RAM
Mitigation: Set sessionExpiryInterval (MQTT 5.0) or a broker-side session timeout. Regularly purge old sessions.
TLS + Authentication Limits
TLS encryption adds CPU overhead on both client and broker. At very high throughput (100K+ clients), TLS handshakes can become a bottleneck.
Authentication (username/password, client certificates) adds latency to CONNECT.
Mitigation: Use connection pooling on the client side; TLS session resumption; IP whitelisting instead of per-client auth when appropriate; separate auth tier (OAuth2 token validation) from the broker.
Implementation Guide
Let’s build a working MQTT system: a sensor that publishes temperature, a subscriber that receives it, and a Mosquitto broker orchestrating them.
Step 1: Install Mosquitto Broker
# Ubuntu/Debian
sudo apt-get update && sudo apt-get install -y mosquitto mosquitto-clients
# Start the broker
sudo systemctl start mosquitto
sudo systemctl enable mosquitto # Auto-start on boot
Verify it’s listening:
netstat -an | grep 1883
# tcp 0 0 0.0.0.0:1883 0.0.0.0:* LISTEN
Default port is 1883 (unencrypted). Next, we’ll enable TLS.
Step 2: Configure TLS/SSL
Generate certificates:
# Create a CA certificate
openssl genrsa -out ca.key 2048
openssl req -new -x509 -days 365 -key ca.key -out ca.crt \
-subj "/CN=mqtt-ca"
# Create a broker certificate
openssl genrsa -out broker.key 2048
openssl req -new -key broker.key -out broker.csr \
-subj "/CN=mqtt.example.com"
openssl x509 -req -in broker.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -out broker.crt -days 365
# Create a client certificate (for testing)
openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr \
-subj "/CN=mqtt-client"
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -out client.crt -days 365
Update Mosquitto config (/etc/mosquitto/mosquitto.conf):
listener 8883
protocol mqtt
cafile /path/to/ca.crt
certfile /path/to/broker.crt
keyfile /path/to/broker.key
require_certificate false
Restart:
sudo systemctl restart mosquitto
Step 3: Python Publisher (Paho MQTT Client)
import paho.mqtt.client as mqtt
import time
import random
# Broker config
BROKER = "localhost"
PORT = 8883
TOPIC = "sensors/temperature"
CA_CERT = "/path/to/ca.crt"
CLIENT_CERT = "/path/to/client.crt"
CLIENT_KEY = "/path/to/client.key"
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to broker")
else:
print(f"Connection failed with code {rc}")
def on_disconnect(client, userdata, rc):
print(f"Disconnected with code {rc}")
def on_publish(client, userdata, mid):
print(f"Message {mid} published")
# Create client
client = mqtt.Client(client_id="temperature-sensor-1")
client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_publish = on_publish
# TLS setup
client.tls_set(ca_certs=CA_CERT,
certfile=CLIENT_CERT,
keyfile=CLIENT_KEY)
client.tls_insecure = False
# Connect and loop
client.connect(BROKER, PORT, keepalive=60)
client.loop_start()
# Publish every 10 seconds
try:
while True:
temp = random.uniform(20, 25)
client.publish(TOPIC, payload=f"{temp:.1f}", qos=1, retain=False)
print(f"Published: {temp:.1f}°C")
time.sleep(10)
except KeyboardInterrupt:
client.loop_stop()
client.disconnect()
Step 4: Python Subscriber (Paho MQTT Client)
import paho.mqtt.client as mqtt
BROKER = "localhost"
PORT = 8883
TOPIC = "sensors/temperature"
CA_CERT = "/path/to/ca.crt"
CLIENT_CERT = "/path/to/client.crt"
CLIENT_KEY = "/path/to/client.key"
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected; subscribing...")
client.subscribe(TOPIC, qos=1)
else:
print(f"Connection failed: {rc}")
def on_message(client, userdata, msg):
print(f"Topic: {msg.topic}")
print(f"Payload: {msg.payload.decode()}")
print(f"QoS: {msg.qos}")
client = mqtt.Client(client_id="temperature-monitor-1")
client.on_connect = on_connect
client.on_message = on_message
client.tls_set(ca_certs=CA_CERT,
certfile=CLIENT_CERT,
keyfile=CLIENT_KEY)
client.tls_insecure = False
client.connect(BROKER, PORT, keepalive=60)
client.loop_forever()
Step 5: C Publisher (libmosquitto)
For embedded systems, libmosquitto (C library) is common:
#include <mosquitto.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
void on_connect(struct mosquitto *mosq, void *obj, int rc) {
printf("Connected with code %d\n", rc);
}
void on_publish(struct mosquitto *mosq, void *obj, int mid) {
printf("Message %d published\n", mid);
}
int main() {
struct mosquitto *mosq = NULL;
int rc;
mosquitto_lib_init();
mosq = mosquitto_new("c-sensor-1", true, NULL);
if (!mosq) {
fprintf(stderr, "Error: Out of memory.\n");
return 1;
}
mosquitto_connect_callback_set(mosq, on_connect);
mosquitto_publish_callback_set(mosq, on_publish);
// TLS
mosquitto_tls_set(mosq, "ca.crt", NULL, "client.crt", "client.key", NULL);
rc = mosquitto_connect(mosq, "localhost", 8883, 60);
if (rc != MOSQ_ERR_SUCCESS) {
fprintf(stderr, "Error: %s\n", mosquitto_strerror(rc));
return 1;
}
mosquitto_loop_start(mosq);
srand(time(NULL));
for (int i = 0; i < 100; i++) {
float temp = 20.0 + (float)rand() / RAND_MAX * 5.0;
char payload[16];
snprintf(payload, sizeof(payload), "%.1f", temp);
mosquitto_publish(mosq, NULL, "sensors/temperature",
strlen(payload), payload, 1, false);
sleep(10);
}
mosquitto_loop_stop(mosq, true);
mosquitto_disconnect(mosq);
mosquitto_destroy(mosq);
mosquitto_lib_cleanup();
return 0;
}
Compile:
gcc -o c-sensor c-sensor.c -lmosquitto
Step 6: Scaling Patterns
For high-throughput scenarios (1M+ msg/sec):
– Use a clustered broker (HiveMQ, EMQX).
– Implement batching on clients (10–100 messages per batch).
– Use topic sharding: different brokers for different topic prefixes.
– Monitor and set per-client quotas to prevent runaway consumers.
For geographic distribution:
– Deploy regional brokers with bridge replication.
– Use clientId to pin sticky clients to a home broker.
– Handle failover with DNS or load balancer.
FAQ
Is MQTT TCP-only?
MQTT 3.1.1 and 5.0 require TCP. However, MQTT-SN (for constrained networks) can run over UDP, Zigbee, or 6LoWPAN. Several vendors (HiveMQ, EMQX) also support WebSocket tunneling of MQTT over HTTP, useful when TCP is blocked by firewalls.
What is a clean session?
In MQTT 3.1.1, cleanSession flag on CONNECT:
– true: Broker discards old session; new subscriptions start fresh.
– false: Broker resumes old session; undelivered QoS 1/2 messages are requeued.
MQTT 5.0 replaces this with sessionExpiryInterval, which sets how long to keep the session (in seconds) after disconnect.
Is MQTT secure by default?
No. MQTT 3.1.1 defines no built-in encryption or authentication. Use:
– TLS/SSL for transport security (encrypts the wire).
– Username/password for client authentication (stored in broker).
– Client certificates for mutual TLS (mTLS).
– OAuth2 tokens (validate externally before CONNECT).
MQTT 5.0 adds AUTH packets for challenge-response, improving flexibility.
MQTT vs Kafka for IoT?
| Aspect | MQTT | Kafka |
|---|---|---|
| Purpose | Message delivery | Event log + streaming |
| Overhead | Tiny (good for constrained devices) | Large (requires robust infrastructure) |
| Retention | Minutes to hours (device-specific) | Days to months (cluster-wide) |
| Partitioning | Topics (wildcard-filtered) | Topics + Partitions (keyed) |
| Scalability | 100K–1M clients per broker | Petabyte-scale data streams |
| Latency | Sub-100ms (RTT) | ~100ms (batching overhead) |
Use MQTT for: Sensor networks, real-time alerts, device control, low-bandwidth IoT.
Use Kafka for: High-volume data pipelines, event sourcing, multi-consumer analytics, durable replay.
They’re complementary: many systems use MQTT to collect from devices and Kafka to stream the aggregated data to analytics.
What is Sparkplug B?
Sparkplug B is a payload format specification (not a protocol) that wraps MQTT messages for industrial automation. It standardizes:
– Message sequencing (seq number prevents out-of-order delivery).
– Payload format (metrics with type, value, quality).
– Node versioning (detect stale node state).
– Timestamps (per-metric and per-message).
Example Sparkplug B payload:
PUBLISH("spBv1.0/factory/DDATA/line-2/sensor-1")
{
"timestamp": 1682074245000,
"metrics": [
{"name": "temperature", "type": "Float", "value": 23.5},
{"name": "humidity", "type": "Float", "value": 65.2},
{"name": "status", "type": "String", "value": "OK"}
],
"seq": 42
}
Sparkplug B is adopted in industrial OPC UA gateways, PLCs, and MES (Manufacturing Execution Systems). It adds safety, but requires careful implementation to maintain sequencing.
Where MQTT Is Heading
MQTT 5.1 (proposed): Enhanced auth flows, improved session management, stricter QoS semantics.
AI-Driven Brokers: Real-time anomaly detection, auto-scaling, intelligent message routing based on content.
MQTT + OPC UA: Convergence of IT (MQTT) and OT (OPC UA). Sparkplug B is the bridge; future tools will make this seamless.
Edge MQTT: Lightweight in-device brokers (femto-brokers) that aggregate before sending upstream. Reduces cloud traffic by 10–100x.
Security Hardening: More mandatory TLS, granular auth (ABAC, RBAC), and audit logging.
MQTT’s 25-year reign isn’t ending; it’s maturing. The 1 billion devices using it today will become 10 billion within a decade. Mastering MQTT is mastering the backbone of IoT.
References
- OASIS MQTT 3.1.1 Standard — https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.pdf
- OASIS MQTT 5.0 Standard — https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.pdf
- Sparkplug B 3.0 — https://github.com/eclipse/sparkplug/blob/main/spec/Sparkplug%203.0.0%20MQTT%20Topic%20Namespace%20and%20Payload%20Format%20for%20Industrial%20IoT%20Applications.pdf
- Eclipse Paho MQTT Clients — https://www.eclipse.org/paho/
- HiveMQ MQTT Essentials — https://www.hivemq.com/mqtt-essentials/
- Mosquitto Documentation — https://mosquitto.org/documentation/
- MQTT-SN Specification — https://www.oasis-open.org/committees/download.php/66091/MQTT-SN_spec_v1.02.pdf
Related Posts
- OPC UA vs MQTT: Bridging IT and OT with Sparkplug B — Understand how OPC UA and MQTT complement each other in industrial settings.
- Unified Namespace Architecture for Industrial IoT — Deep dive into ISA-95 and Topic-Based Addressing for enterprise IoT.
- A Technical Comparison of Major IoT Protocols: Choosing Your Protocol — MQTT, CoAP, Zigbee, LoRaWAN, NB-IoT side-by-side.
- CoAP Protocol: Constrained Application Protocol for IoT Deep Dive — Lightweight alternative to MQTT for resource-constrained devices.
Last Updated: April 18, 2026
