The idea: a broker in the middle, and QoS per hop

MQTT is a small publish–subscribe protocol for sensors and phones. Every client keeps one TCP connection to a broker (port 1883). A publisher sends a PUBLISH with a topic such as home/kitchen/temp. A subscriber has sent SUBSCRIBE with a topic filter. The broker forwards each PUBLISH to every client whose filter matches. Publisher and subscriber never talk to each other.

Three ladder diagrams between a sender and a receiver. QoS 0: one PUBLISH, no reply. QoS 1: PUBLISH id 1, then PUBACK id 1; the sender keeps a copy until the PUBACK. QoS 2: PUBLISH id 1, PUBREC id 1, PUBREL id 1, PUBCOMP id 1; the receiver stores the message under id 1 until PUBREL
QoS 0 sends once, QoS 1 waits for PUBACK, QoS 2 needs four packets so the receiver can spot a resent copy by its packet id.

The page has one publisher, sensor-1, and one subscriber, app-1. Each quality of service (QoS) level holds on one hop: sensor-1 → broker, and again broker → app-1. The broker sends to app-1 at the lower of the publish QoS and the QoS app-1 asked for in its subscription. Each side picks its own packet identifiers.

This page is about MQTT's packets. For publish–subscribe in general, see Observer vs Publish–Subscribe. For at-most-once, at-least-once and exactly-once in general, see Delivery Semantics.

CONNECT and SUBSCRIBE

A client starts with CONNECT. It carries the client id, Clean Session (0 = keep my session while I am away), Keep Alive (the longest gap between packets, in seconds) and, if wanted, a will. The broker answers CONNACK, whose Session Present flag says whether it still had a session for that id. Then the subscriber sends SUBSCRIBE with a packet id, filters and a QoS, and the broker answers SUBACK with the QoS it granted (Demo: connect, subscribe with +, publish).

Topics and wildcards

Topic levels are split by /. In a filter, + matches exactly one level and # (last, alone) matches that level and everything below it. A publisher always names a full topic; only filters have wildcards.

filterhome/kitchen/temphome/kitchen/humidityhome/temp
home/+/temp✓✗✗ (no middle level)
home/#✓✓✓
home/kitchen/+✓✓✗
+/+/temp✓✗✗

A lost packet is a broken connection

MQTT runs over TCP, which already resends lost segments. So a packet that never arrives means the connection broke and took the packet with it. MQTT 3.1.1 and 5 resend only at one moment: after the client reconnects with Clean Session 0 and the broker says Session Present 1. Then the client resends every unacknowledged PUBLISH with DUP = 1 and the same packet id, and every unanswered PUBREL. Each demo loses one packet on the sensor-1 → broker hop.

QoS 0: at most once

One PUBLISH, no packet id, no reply. The sender keeps nothing. If the connection breaks with the PUBLISH on the wire, there is nothing to resend after the reconnect, and the reading is gone (Demo: QoS 0, the PUBLISH is lost). It is the cheapest level. It fits readings that the next reading replaces.

QoS 1: at least once

The sender keeps the message in its in-flight list until PUBACK comes back with the same id. The broker forwards as soon as the PUBLISH arrives, then sends the PUBACK. If the PUBACK is lost, sensor-1 cannot tell whether the PUBLISH arrived. After the reconnect it resends PUBLISH id=1 with DUP=1. The broker treats it as a new publication and forwards it again, so app-1 gets 21.5 twice (Demo: QoS 1, the PUBACK is lost). The app must cope with repeats, for example by being idempotent.

QoS 2: exactly once

Four packets. PUBLISH: the broker stores the message under its packet id and does not forward it yet. PUBREC ("received"): the sender can now drop the message and keeps only the id. PUBREL ("release"): the broker forwards the stored message and forgets the id. PUBCOMP ("complete"): the sender frees the id.

Time runs down. sensor-1 sends PUBLISH id 1 QoS 2; the broker stores it under id 1. The PUBREC is lost and the connection breaks. sensor-1 sends CONNECT with Clean Session 0, the broker answers CONNACK Session Present 1. sensor-1 resends PUBLISH id 1 with DUP 1; the broker finds id 1 already stored, keeps one copy and sends PUBREC again. Then PUBREL, the broker forwards once to app-1, and PUBCOMP
With the PUBREC lost, the resent PUBLISH finds id 1 already stored, so the broker keeps one copy and forwards the reading once.

Each loss ends the same way. A lost PUBREC makes sensor-1 resend the PUBLISH, and the broker sees id 1 is already stored (Demo: QoS 2, the PUBREC is lost). A lost PUBREL is simply resent. A lost PUBCOMP makes sensor-1 resend the PUBREL. The broker has already forgotten id 1, answers PUBCOMP anyway, and forwards nothing again (Demo: QoS 2, the PUBCOMP is lost). The broker → app-1 hop runs the same four steps.

Retained messages

A PUBLISH with RETAIN = 1 is also kept by the broker as the topic's retained message, one per topic: a new one replaces the old one. When a client subscribes, the broker sends it each matching retained message at once, with RETAIN = 1. A sensor that reports only on change can still tell a new dashboard its last value (Demo: retained message for a late subscriber: 40 then 41 are retained on home/kitchen/humidity, and the late subscription gets 41). An empty retained PUBLISH deletes it.

Last will and keep alive

In CONNECT a client can leave a will: a topic, payload, QoS and retain flag. The broker publishes it when the connection ends without a DISCONNECT. If TCP breaks, that is right away. If the client just goes silent, as when a sensor loses power, the broker waits 1.5 × Keep Alive with no packet (15 s here) before it closes the connection. In Demo: last will when the sensor dies, app-1 is subscribed to home/+/status and gets home/kitchen/status = offline, which also stays retained for later subscribers. A clean DISCONNECT deletes the will.

Sessions: coming back after a disconnect

With Clean Session 0, the broker keeps a client's session after it leaves. The session holds its subscriptions, its unfinished QoS 1 and 2 handshakes, and a queue of QoS 1 and 2 messages that arrived while it was away. On the next CONNECT the broker answers Session Present 1 and sends the queue. QoS 0 messages are not queued (Mosquitto's default). In Demo: offline subscriber, session kept, 21.5 (QoS 1) and 7.0 (QoS 2) wait in the queue, and 22.0 (QoS 0) is dropped. With Clean Session 1, the session is thrown away at disconnect and every message from that time is missed.

Which QoS?

QoS 0QoS 1QoS 2
packets per hop124
sender keepsnothingthe message, until PUBACKthe message until PUBREC, then the id until PUBCOMP
a lost packet giveslossmaybe a duplicateexactly one copy
queued for an offline sessionnoyesyes
good forfrequent readingscommands and events the app can repeat safelythings that must not repeat, such as billing counts

QoS 2 is "exactly once" for one hop. If the app crashes after the broker hands it a message, it is still up to the app not to act twice.

What MQTT 5 changes

MQTT 5 renames Clean Session to Clean Start and adds a Session Expiry Interval, so a session can live for a set time. It adds a Will Delay Interval, so a quick reconnect does not fire the will. Acks carry reason codes, such as "no matching subscribers". The QoS handshakes on this page are the same in both versions.

See also the TCP handshake for the connection MQTT runs on, and Kafka for a log-based broker that keeps every message instead of one retained message per topic.

What the page leaves out

TLS (port 8883) and username/password or client certificates; PINGREQ/PINGRESP (the keep-alive packets an idle client sends); UNSUBSCRIBE; several subscribers and shared subscriptions ($share/…, MQTT 5); topic aliases, message expiry and user properties (MQTT 5); flow control (Receive Maximum) and the limit on in-flight messages; QoS 2 "method B", where a receiver forwards at PUBLISH and keeps only the id; packet loss on the broker → app-1 hop, which works the same way; bridging brokers; and how a broker stores sessions on disk.

On the mind map

Open the Distributed Systems mind mapDistributedSystemsReplication: copies of data · what reads seeReplicationPartitioning: split data over nodesPartitioningTime and membership: order events, know who is aliveTime and membershipConsensus: agree on one value or logConsensusTransactions: one change across many servicesTransactionsMessaging: send events between systemsMessagingKafka internals: a replicated append-only logKafka internalsRedis internals: one thread, data in memoryRedis internalsKubernetes: reconcile desired stateKubernetesResilience and speed: stay up and fast under loadResilience and speedDelivery semanticsDelivery semanticscommit before or after processingMQTT QoS levelsQoS 2: PUBREC/PUBREL drop resent copiesPush vs pullPush vs pullbroker pushes with prefetch vs pollChange data captureChange data capturetail the WAL, publish each row changeETL and SCD typesETL and SCD typesType 2 = new row, new key, history keptRedis StreamsRedis Streamsconsumer group, PEL until XACKLog vs queueLog vs queuequeue deletes on ack; log keeps offsetsOpen the Distributed Systems mind mapDistributed Systemssend events between systemsMessagingDelivery semanticsDelivery semanticscommit before or after processingMQTT QoS levelsQoS 2: PUBREC/PUBREL drop resent copiesPush vs pullPush vs pullbroker pushes with prefetch vs pollChange data captureChange data capturetail the WAL, publish each row changeETL and SCD typesETL and SCD typesType 2 = new row, new key, history keptRedis StreamsRedis Streamsconsumer group, PEL until XACKLog vs queueLog vs queuequeue deletes on ack; log keeps offsets
This page on the Distributed Systems mind map · 82 pages in 10 branches · open it to see the whole map and test yourself in recall mode.