There are numerous reasons why a permanent sensor-to-cloud data stream is not a good idea for many applications. Increasing data volumes, insufficient bandwidth, long latency times, data protection issues and relatively high operating costs are just a few examples. Despite all this, the cloud has established itself as a data processing platform for IoT data in recent years because it offers highly developed, user-friendly and well-documented tools and options for data processing. The possibility of location-independent user access to the data stored or generated in the cloud is also particularly important. Furthermore, a cloud offers almost unlimited resources in terms of storage capacity and computing power, which are also very professionally managed and can even automatically balance out load fluctuations.

Edge AI (AI: Artificial Intelligence) takes a completely different approach: data should be processed directly on site in order to generate the required information - often even in real time - and use it for automatic decision-making.

However, edge AI differs from edge computing, which is often referred to as edge cloud deployment. This usually involves simply using cloud services on correspondingly powerful hardware directly on-premises. In some cases, the same software interfaces (APIs) are even used as in the respective cloud applications. Edge computing is primarily a hardware issue. From an expert's point of view, the right edge computer is crucial here.

Edge AI pursues a slightly different objective. It is primarily about using powerful AI algorithms in even the smallest microcontroller for real-time data analysis and automatic decision-making. Hardware and software must be carefully coordinated for such solutions.

However, various hybrid solutions are also possible. In such a case, the edge AI solutions send the metadata generated by machine learning algorithms directly to a cloud. Alternatively, an Edge AI sensor can act as a modular solution by being equipped with a data interface to an Edge Gateway, which in turn runs a Docker container belonging to the sensor with the data analysis algorithms.

However, a very dynamic Edge AI development can initially be expected in the area of wireless IoT sensor technology with cloud integration, particularly in terms of innovation, application diversity and the need for suitable semiconductor chips.

Edge AI application wireless sensor technology

A typical IoT application consists of three elementary functional blocks: the sensors, the gateway functions and a cloud, for example as an IoT platform with various services. Until now, sensors and gateways were primarily designed as data providers for the cloud. Now, with Edge AI, another demanding task is being added: this includes real-time sensor data analysis directly on site at the sensor data source (for example as a machine learning inference via TensorFlow Lite).

© SSV Software Systems

The system integration of wireless sensors in IoT solutions is particularly demanding in industrial applications. The respective protocol stack is still a major challenge. It must not only be available for the sensor itself, but also for the gateway to be used. In addition to the functions and interfaces, the respective license models must also be taken into account. Even the developers behind the code should be evaluated as part of the decision-making process. After all, an industrial wireless sensor solution also requires qualified support for the protocol software for the next 10 to 15 years. A wireless firmware as a binary object (blob) from a non-transparent source, as is common for Wi-Fi or BLE implementations in the consumer sector, is not always a good choice.

Another important issue for the wireless sensor technology of an edge AI solution is IT or IoT security. First and foremost, this includes the authenticity, confidentiality and integrity of the sensor data. An IoT application at the other end requires 100% certainty that the data received actually originates from the sensor whose sender ID can be found as the source in the sensor data and that this data has not been modified during transmission. With the required end-to-end security, it is often overlooked that the security mechanisms of a TLS protocol used by MQTT or HTTPS connections are 'broken' at more than one point in practice due to the architecture. As a result, there are several areas between the sensor and the application where sensor data can be manipulated or other unwanted interventions can take place. However, a blockchain is not immediately necessary to eliminate such risks as far as possible. In practice, it is sufficient if the sensor data is provided with a digital signature directly at the source or in the immediate vicinity, which is verified by the application at the other end in any case.

OTA updates for sensor and gateway

Another security issue is the technical and organizational requirements for over-the-air (OTA) updates. A unidirectional communication path for transmitting measurement data from the sensor to the cloud is therefore no longer sufficient. In the opposite direction, i.e. from the cloud to the sensor, software updates must be sent from time to time or as required. This is a highly sensitive task due to the risk of misuse.
Reliability aspects must also be taken into account here: If the wireless data transmission between the sensor and gateway is interrupted during an update process, the IoT sensor must still be able to use the 'old' software version. To do this, the internal flash of the sensor microcontroller should be divided into an A and B area in order to apply the A/B boot update concept. The software is started from one area at a time. The update is carried out in the other area. Only when a software update has run through completely and without errors are the areas switched and a restart of the microcontroller software triggered in order to use the new software version.

The gateway functions of an application are integrated into the communication relationships for software updates. They not only serve as an OTA update proxy for the sensor update, but also require function and security updates from the cloud themselves. By using ML algorithms (i.e. the AI part) to analyze sensor data in real time within the gateway functions, updates are also required for the machine learning model used (ML model updates). Such edge AI solutions use models that are created through a machine learning training phase (ML model building) in the cloud.

The AI part

The ML algorithm of an Edge AI application can be imagined as a mapping function in the form of a neural network: the appropriate output values Y are supplied for the respective input parameters X via a mathematical regression or classification process. The mathematical relationships for the arithmetic operations in the individual nodes (the neurons) are learned from previously recorded training data as part of ML modeling.

© SSV Software Systems

The artificial intelligence of an Edge AI IoT application is created using supervised machine learning methods. The sensor then includes an ML model for the inference phase - for example, a real-time data analysis using a previously learned pattern recognition that matches the respective task. The ML algorithm used for this can be imagined as a mapping function: the appropriate output values Y are supplied for the respective input parameters X via a mathematical regression or classification process. The relationship between X and Y is learned by the mapping function from previously recorded training data. The mathematical mapping procedure can be implemented using a neural network.

Due to the numerous iterations, ML modeling with a neural network is a computationally intensive process that should be carried out with a sufficiently large amount of training data. In addition, an ML model requires a few additional learning phases from time to time in order to react to changes in the inference environment and optimize the inference error rate. Furthermore, the current model version must be centrally available to all sensing endpoints of an edge AI application. Due to these different requirements, a cloud or an equivalent on-prem service is the best place for model creation, maintenance and model storage in most cases.

A new embedded megatrend?

Klaus-Dieter Walter is a member of the management board at SSV Software Systems.

© SSV Software Systems

Edge AI and the decentralized artificial intelligence associated with it could become something of an embedded megatrend in the IoT world, as it allows semiconductor manufacturers, for example, to achieve gigantic quantities. Companies such as ARM are already talking about 'trillions of intelligent endpoints' with wireless sensor technology and local endpoint AI. At the virtual embedded world 2021 Digital trade fair, the first products that give an idea of what is meant by this could already be discovered - the 'digital nose with AI' is just one example.