Today, almost anything can be connected to a wireless network and linked in the Internet of Things. Everyday objects such as cars, televisions, wristwatches, heaters, coffee machines and even umbrellas can now use an internet connection that was previously reserved for laptops and smartphones. By 2020, around 20 billion networked devices will be in use in the IoT.

At this scale, conventional client-server software architectures and communication protocols often reach their limits on the server side. On the client side, on the other hand, poor connection quality is often to be expected, especially when data is transmitted via mobile communications.

The MQTT protocol was developed back in 1999 by IBM and Arcom Control Systems as part of a joint project to monitor oil pipelines. The aim was to create a bandwidth-efficient and battery-saving protocol. In 2010, IBM published the license-free MQTT 3.1 specification, which gave rise to many proprietary and free software products on both the server and client side.

A formal standardization process began in 2013, which made MQTT 3.1.1. an open and freely available ISO (ISO/IEC 20922: 2016) and OASIS standard. At the end of 2015, work began on a comprehensive update of the MQTT standard - MQTT version 5. Based on user feedback, MQTT 5 extends the protocol with important functions that enable modern, cloud-based IoT applications and also became an OASIS standard in March 2019, which is supported by commercial and open source implementations.

MQTT has since become one of the most popular IoT standard protocols for connecting devices in the Internet of Things. Due to its leanness and simplicity, it can be used in many projects where other protocols such as OPC UA are too heavyweight or too complex. Above all, however, MQTT provides answers to issues such as scaling, reliable data transmission over unreliable networks and standardized security measures.

Publish/Subscribe architecture

The push principle of MQTT: It enables highly efficient communication between participants.

© HiveMQ

The MQTT protocol implements the publish/subscribe approach. The producers of messages (publishers) and the recipients of the messages (subscribers) are completely decoupled by an MQTT broker. This MQTT broker is responsible for delivering a message to the correct recipient and manages the individual connected clients. The broker is able to send a single message from a publisher to a large number of subscribers. Both sending and receiving clients remain constantly connected to the MQTT broker via a standing TCP connection.

The MQTT broker can therefore deliver a message to interested clients using the push method without delay. The sending (publish) and receiving (subscribe) of messages works via so-called topics. A topic is a string that represents a type of message subject, but is structured similarly to a URL.
The structure 'heimautomatisierung/peters_haus/wohnzimmer/gluehbirne/1', for example, would be a topic for messages from light bulb 1 in the living room of Peter's house.

Every MQTT client that wants to receive messages for this topic subscribes to it at the broker. The interested clients are notified by the broker when new messages arrive, instead of having to ask the server for changes themselves. This push principle enables highly efficient communication between the participants. The decisive factor is that the participants in the communication do not need to know anything about each other. Each client only needs to know the message broker and can still participate in communication with all clients.

Essential protocol features

In addition to the exchange of messages, its wealth of features makes the MQTT protocol ideal for the Internet of Things, especially if an unreliable connection - for example via mobile radio - is to be expected. MQTT has the concept of quality of service levels (QoS levels). These QoS levels define the transmission guarantees for messages. A distinction is made between three types of QoS:

QoS 0At-most-once delivery. An attempt is made to deliver the message once. The message reaches the recipient once or not at all.
QoS 1At-least-once delivery. The sender resends the message if the recipient does not acknowledge receipt.
QoS 2Exactly-once delivery. The MQTT protocol ensures that the message reaches the recipient exactly once.

MQTT uses TCP as the transport protocol, which normally provides 'Exactly-once Delivery' guarantees during data transmission. However, it can happen at any time
happen that the TCP connection is disconnected during data transmission. In such a case, QoS 1 and QoS 2 take effect, as data transmission can be continued after the connection has been re-established.

Another protocol feature of MQTT is 'retained messages'. A publisher can mark an MQTT message as retained when it is sent. The MQTT broker now saves this message for this topic so that all new subscribers on this topic receive the message directly. With a normal MQTT message, the message would be sent to all active MQTT subscribers, but nothing would be stored at the broker. A new subscriber would then have to wait until a new message is sent on a topic. Using retained messages, each new subscriber now receives the latest status on a topic, provided that the retained flag for a message to be sent has been set by the client.

In use cases where clients are likely to disconnect frequently, the broker can maintain a persistent session. When a client reconnects, the broker sends all missed messages for its subscriptions to the client. In addition, the client does not have to re-subscribe to previously subscribed topics, as the broker simply continues the previous session.

Why MQTT for IoT?

Dominik Obermaier is CTO and co-founder of HiveMQ.

© HiveMQ

MQTT is particularly suitable for reliable message transmission in unreliable and unstable networks, such as mobile networks. This makes MQTT an ideal protocol for the Internet of Things and mobile communication. MQTT is completely data agnostic and can transmit data of any type, whether it is text or binary coded content. The concepts of the protocol are easy to learn and custom client implementations are straightforward to realize. MQTT is based on TCP and the transmission can be encrypted at any time using SSL/TLS. Unlike other protocols, there is no polling with MQTT. Messages are distributed immediately when an event occurs, as MQTT enables genuine push communication. This saves bandwidth and CPU, as MQTT client applications can react as soon as a message arrives instead of asking the server for new messages.

With the help of MQTT, use cases can be implemented in all industries and sectors: Car manufacturers or car-sharing applications rely on MQTT for reliable connectivity and continuous availability. This guarantees the user that commands to the car-sharing service or the vehicle itself are executed quickly and reliably throughout the entire journey - regardless of the user's location or demand at peak times.

MQTT is also relied on by many manufacturers who need a reliable messaging platform for connectivity between a networked factory and the cloud. This enables companies to network their production facilities around the world and collect and process the generated data in a central backend.

MQTT 5

MQTT 5 is the largest and most feature-rich update to the MQTT protocol ever, while retaining the principles that have made it the most popular protocol for the Internet of Things to date.
Features of the MQTT protocol version 5, include: Simplifications of the protocol for highly scalable systems, improved error handling and reporting, extensibility of the protocol, simplifications of state transitions, identification and provision of functions for common communication patterns with MQTT and improved authentication and authorization mechanisms.
Version 5 thus eliminates criticisms of its predecessor version and offers new functionalities that make it even easier to implement typical IoT use cases.

First steps

The MQTT broker is the heart of communication with MQTT. It is therefore important that the MQTT broker can meet the requirements of the use case. In addition to performance, security features and the robustness of the broker also play a role. There is now an abundance of MQTT brokers:

● Open source solutions such as mosquitto or the HiveMQ Community Edition.

● Commercial MQTT brokers such as HiveMQ Professional or Enterprise Edition, which can network up to several million MQTT clients in a cluster.

● Cloud-based MQTT brokers such as HiveMQ Cloud, for which no self-provisioning of the MQTT broker is required.

For an easy introduction to the MQTT protocol, there are also all kinds of tutorials and resources on the Hive MQ website.