DataArt
Does it always have to be MQTT?
When a company builds a platform for the collection, provision and storage of device data and the development of algorithms for machine learning, the question of a smart choice of architecture always arises. Some project examples.
There are many providers of cloud solutions that provide users with all the necessary tools - flexible and universally applicable. But how can users keep the solutions simple and reduce deployment costs? Is it always cost-effective to follow standard guidelines? Is it possible to save money and even simplify projects by deviating from the standard architecture?
Suppose you are building an Internet of Things project and there is no in-house experience. What should be done? After thoroughly weighing up all the pros and cons, an existing IoT platform seems to be a good solution. There are a number of guidelines issued by experts.
MQTT - the favorite
It is obvious that the guidelines are created in the most universal and best applicable way. The MQTT protocol can be found in most of them. AWS IoT, Azure IoT and Google Cloud IoT are based on MQTT messages. A popular open source message broker, Eclipse Mosquitto, also uses MQTT for message exchange.
An example: In a project realized by DataArt in the field of heavy industry, data from sensors had to be delivered from the field. It was to be used in a digital twin solution to predict metrics and values of possible future conditions. The solution was built on AWS. Figure 1 shows the original architecture offered.
However, such architectures can often be simplified. This can be seen in another project involving thermostats and HVAC systems. Here it is practical to send and receive commands via the same channel and even automate some logic on the MQTT broker side. When the thermostat changes state, an MQTT message can be fetched directly from the HVAC device that has subscribed to the corresponding MQTT topic. As these devices are connected to a local gateway that also acts as an MQTT broker, the entire system can continue to operate without a connection to the cloud, i.e. completely locally. However, such solutions require an advanced firmware upgrade system for all devices in order to make changes to the logic of the devices.
To simplify the example even further, it is possible to imagine a project in which MQTT-capable light bulbs are to be controlled with MQTT-capable wall switches (Fig. 2) . And according to the requirements, it is sufficient to control them locally. All that is needed is an MQTT broker. Subscriptions can thus be created between light bulbs and switches to describe the logic with which the light bulbs are controlled by the switches.
But now back to the industrial project with the AWS architecture described at the beginning: there was a need to collect data for the digital twin that performs some calculations. Was MQTT an optimal choice?
Sensors only generate data. There is no need to send anything to the sensors and between the sensors. Do we really need all the features of MQTT? Is it really necessary to pay fees for AWS IoT and AWS Lambda services and use the MQTT brokers?
Figure 3: Only the costs for the data lake are incurred. The MQTT brokers do not need to be maintained. This is a double win situation.
© DataArtSave AWS costs
Everything can be made a little easier by sending the data directly to a data lake. Since
AWS GreenGrass is already in use, there are credentials for AWS and the boto3 library on the gateway side. So why not exclude MQTT and Lambdas from this project to save AWS costs? You can give the data lake the "Insert only" authorization and simply insert the data from the gateway side. The architecture is shown in Figure 3.
The calculations for 10,000 devices now look like this: Each device sends a message once per minute. In the case of AWS IoT, the connection costs must be paid: 432,000,000 connection minutes correspond to 34.38 dollars per month. Then there is the fee for the messages: 432,000,000 messages per month is 432 dollars. And the payments for AWS Lambda calls, which in the case described amount to 288 dollars. In total, you can save 750 dollars per month for 10,000 devices. And that's even without further DevOps optimizations, simply because the right decision was made in the design phase of the solution.
Room for optimization
Most digital twin/IIoT projects only need to collect data. This allows simpler protocols such as direct access to the data lake or even HTTPS requests to be used to deliver such data. However, the choice always depends on the requirements. There are no one-size-fits-all solutions and there is always room for optimization.
Reducing operating costs is a trend in modern IIoT innovations. Above all, this shows that the industry has reached this stage of maturity. The introduction of innovations requires detailed planning, thorough ROI calculations and a deep understanding of every aspect of an evolving solution. It's important to realize what a 20-30% OpEx difference would mean to your business - and that 20-30% savings is just the average overhead DataArt has seen in recent optimization projects. So no, it's not always about MQTT.

















