Artificial intelligence is increasingly shaping the world of embedded systems - beyond image processing. The question of "whether" has long since been answered; instead, the focus is on "how" - and therefore on complex questions regarding suitable cloud and/or edge concepts, hardware and software architectures and project approaches.

The crucial question: Cloud or Edge?

Overview of embedded AI hardware and its fields of application.

© Grossenbacher systems

Since ChatGPT, it can be observed in the market that developers are once again increasingly relying on the processing of unfiltered machine data in the cloud. This is understandable given the advantages of ChatGPT or other AI engines and the available resources in cloud-based data processing, but at the same time doubly questionable: firstly, mass data transmission consumes energy, and secondly, every query, i.e. every "call", also costs money.

To minimize communication, algorithmic pre-processing of data at the edge level is often the more sensible approach. The amount of data required and the number of data sources are decisive factors here. If the data volumes are very large, the question arises as to which should be forwarded directly to the cloud and which should first be processed and summarized on site using suitable AI algorithms.

Latency also plays a role: how long can my application wait for the AI to react? The answers to these questions determine what level of intelligence and therefore what hardware makes sense at the edge level. Depending on the scope of the algorithm calculation in the cloud or the edge, different hardware architectures are recommended - they range from simple microcontrollers with AI extensions "on chip" to FPGAs, DSPs, TPUs and powerful and expensive additional computers with GPUs. All variants have specific strengths and limitations that predestine them for certain AI applications (see box "Hardware - What is the smartest choice?").

In addition to the hardware, the software and the development process also raise questions. For AI applications in automation, the required response times are a critical issue. If the AI application relies on fast responses, the algorithms must be processed locally. The more complex these are, the larger the data volumes and the shorter the desired response times, the more computing power is required. This incurs costs - both for development and for the edge hardware itself. They must fit the business model and provide the end customer with a benefit that justifies the additional costs. No technology can prevail without proof of economic viability.

Nevertheless, the use of AI engines such as ChatGPT is not necessarily more cost-effective - "pay per call" sends its regards. There are several factors that determine the distribution of embedded AI and "normal" AI. If embedded is used at least in part, the product owner must decide on the software development toolchain - depending on the hardware used, including the AI processors, of course.

The development and provision of AI applications requires tools and libraries that are suitable for the selected hardware platform.

Model compression techniques are hugely important in reducing hardware requirements. There are no "one size fits all" software solutions for all hardware. However, there are some platforms and frameworks that support different types of hardware and facilitate the development and implementation of AI applications on a variety of devices (see box "Software - Who can work with whom?").

AI "lives": Fleet Observability

AI algorithms are never error-free and can always be optimized. In addition, the algorithms "drift", i.e. the quality of the results they deliver - for example when classifying sensor data and detecting anomalies - can decrease over time due to environmental influences. AI algorithms therefore require continuous testing and adaptation.

To ensure the quality and safety of AI in the embedded environment, comprehensive diagnostics and monitoring of all installed AI field devices is therefore required - in other words, "fleet observability". This can be achieved by a centralized monitoring solution with AI support that is able to process diagnostic data, react to it and trigger alarms and reports to support the embedded AI application as a whole. For this purpose, there is a cloud portal solution that can take over the diagnostic and management tasks for the AI field devices and their algorithms; this also includes updating the algorithms themselves.