In the Gartner Hype Cycle for Emerging Technologies, platforms for the Internet of Things have almost reached the Peak of Inflated Expectations. Nevertheless, there is still no generally accepted definition for an IoT sensor or an IoT sensor system. An internet search with the string 'IoT sensor' brings up tens of thousands of hits, but also illustrates the wide range of technologies and providers in this area. In some forums, for example, the terms 'smart sensor' and 'IoT sensor' are linked together, although a 'smart sensor' can also have very different properties depending on how you look at it. Examples of IoT sensor systems range from 'Body Sensor Network for Healthcare Systems' and 'Industrial Wireless Sensor Networks for Data Collection' to the almost unmanageable variety of special solutions from various providers for vertical markets.

Standards, such as the IEEE 1451 collection with its various sub-standards, are obviously outdated from an IoT perspective and therefore practically irrelevant in this area. Not even the very useful ideas of the IEEE-1451.4-based Transducer Electronic Data Sheet (TEDS) have been given any consideration in IoT applications to date. One reason for this is probably the completely different speeds of innovation in the Internet of Things and the IEEE as an international standards body. An attempt to create a corresponding IoT sensor standard via a new provider consortium to be founded in 2016 has obviously not gone beyond the dissemination of a press release.

The AMA Association for Sensor Technology in Germany offers guidance on universal smart sensors, albeit at a very early stage. In the 'Sensor Technologies 2022' study published in mid-2018, the authors describe the functional units of a smart sensor and provide examples of how such a sensor can be implemented in principle. With minimal extensions, the AMA view can be used as a template for a generic IoT sensor. In the study, the AMA also discusses the importance and functions of an embedded system in smart sensors.

The AMA document assumes that the entire firmware of a smart sensor is created as part of a conventional embedded software development process. This assumes that the relationship between the sensor input variables (sensor element input) and the desired output value range is fully known at the start of development and can also be coded for an embedded system by the software developers involved. This approach may still work with simple sensor elements such as temperature, humidity and pressure and a manageable sensor fusion. However, it will not work for future smart sensors that use descriptive and predictive data analyses or neural networks to reliably recognize complex states and patterns from the raw data of various sensor elements in near real time and derive output values from them. Due to their complexity, sensors are then created that only transmit raw data directly to edge or cloud systems using suitable protocols (sensor-to-cloud solutions with sensor data streaming). The actual acquisition of information is shifted to other levels because powerful computer platforms and corresponding services are available there. However, this requires correspondingly fast communication connections (Ethernet, 4G, in future 5G), a powerful energy supply and a continuous expansion of the internet bandwidth. A serious disadvantage is that the information obtained from the sensor data is not available directly at the communication interface of the embedded system, but in the edge or cloud. Furthermore, this method is not suitable for time-critical sensor data evaluations.

Artificial intelligence embedded

Thanks to spam filters in email programs and the voice assistants from Amazon, Apple, Google, Microsoft and other providers, products with integrated algorithms from the field of artificial intelligence (AI) have gradually become mass applications. Researchers now even regard the development of AI as a universal technology and place it on a par with the inventions of the steam engine, electricity and the combustion engine in terms of its expected impact. Not only self-driving trucks and cars, but practically everything that contains at least one microchip will also have AI-based functions in the future. The same applies to application software: sooner or later, every smartphone app, PC spreadsheet or enterprise IT application will contain AI algorithms - in many cases, this is already state of the art. This development will also permanently change the properties of sensors.

The current state of the art for IoT sensor systems is to transmit sensor data as a stream to a cloud via the internet using special protocols such as REST, MQTT, CoAP and LWM2M and to process it there. Millions of such sensor-to-cloud applications with conventional sensors are now in use around the world. Depending on the platform provider, different services are available in the cloud to obtain the desired information from the sensor data. Machine learning and deep learning algorithms are also used to classify raw sensor data, for example. The results, i.e. the output data of a classification algorithm, are often sent via the internet to an IoT device that is located in close proximity to the sensor data source. A typical example would be a sensor-actuator combination in a cyber-physical building automation system where the sensor and actuator are located in the same building, but the AI algorithm used by a cloud service is located on a server several thousand kilometers away. Both sensor and actuator are connected to the internet via 4G mobile communications, for example. If you now imagine that a 32x32-bit RGB image sensor is used as the sensor element to reliably identify ten different objects using the classification algorithm and influence the environment according to the identified object, it quickly becomes clear that a total of 32x32x3 bits = 3072 bits plus protocol overhead are transmitted from the sensor to the cloud for each object detection, although the discrete result, i.e. an object ID from the range 1 to 10, can be represented using 4 bits. We are therefore dealing here with a conceptually induced 768-fold net data overhead per transaction. The overall very inefficient use of the transmission channel is clearly visible in this sensor-to-cloud example.

The data overhead and other reasons have led to sensor-to-edge solutions and fog computing becoming established alongside sensor-to-cloud solutions. This involves installing a special computer system (edge gateway) for sensors, actuators and other devices within a local environment that can be accessed directly by all devices, i.e. without internet access. The same machine learning and deep learning algorithms are used on the edge gateway as in the cloud. The analysis and evaluation or assessment of the IoT sensor data, for example with the help of statistical learning (classification, regression, prediction of probabilities, neural networks for data classification) is then not carried out in the distant cloud, but in the immediate vicinity of the data sources.

An AI algorithm enables sensors to evaluate the raw sensor data of the sensor elements in real time using a machine learning model or neural network and provide specific information at the output.

© SSV Software Systems

Classification and regression algorithms are established components of supervised machine learning and deep learning in connection with convolutional neural networks (CNNs, also known as convolutional neural networks) and binarized neural networks (BNNs). These algorithms can also be embedded directly in an IoT sensor and used without a connection to the cloud and edge gateway. With CNNs, however, it must be taken into account that the deep nesting (CNN hidden layer) of these artificial neuron networks sometimes requires considerable computing resources in the embedded system of the IoT sensor. Deep convolutional neural networks with numerous hidden layers, such as those used by Google and others with TensorFlow for image and speech recognition, are currently unsuitable for direct use in sensors. Unsupervised machine learning is also less suitable for direct sensor use. The algorithms available for this try to recognize previously unknown patterns in the input data. However, this automatic segmentation (clustering) can play an important role in preparing the practical use of an AI-based IoT sensor.

The bottom line is this: Sensor technology is undergoing major changes due to the IoT. Generally applicable specifications are urgently needed for sensor manufacturers and users alike. However, it is not enough to simply declare existing smart sensor and sensor-to-cloud solutions the standard. There is an urgent need to include other topics, such as artificial intelligence and state-of-the-art security.

Author:
Klaus-Dieter Walter is a member of the management board at SSV Software Systems.