IoT applications are generating more data than ever before. For many industrial applications, especially highly distributed systems in remote regions, it may not be possible to constantly send large amounts of raw data to a central server. To reduce latency, lower the cost of data communication and storage, and increase network availability, companies are turning to artificial intelligence (AI) and machine learning to make decisions and take actions in the field in real time.

These cutting-edge applications using AI capabilities in IoT infrastructures are referred to as "AIoT" applications (Artificial Intelligence of Things, AIoT). While AI models still need to be trained in the cloud, data collection and inferencing can be done on-premise by deploying trained AI models on edge computers. This article describes how to choose the right edge computer for your industrial AIoT application.

AI integration into the IIoT

The number of industrial devices connected to the internet continues to increase year on year and is expected to reach 41.6 billion endpoints by 2025. Manually analyzing the information generated by all the sensors on an assembly line, for example, is almost impossible. It's hardly surprising that "less than half of a company's structured data is actively used for decision making - and less than 1% of their unstructured data is analyzed or used at all" [1]. Artificial intelligence and machine learning help to leverage this untapped potential.

In the industrial application described above, AIoT offers the opportunity to reduce labor costs, reduce human error, for example in a manual quality inspection, and optimize preventive maintenance. The 'Artificial Intelligence of Things' (AIoT) refers to the use of AI technologies in Internet of Things (IoT) applications to improve operational efficiency, human-machine interactions and data analysis and management [2]. But what exactly is AI and how does it fit into the Industrial Internet of Things (IIoT)?

'Artificial intelligence' as a general field of science deals with the construction of intelligent programs and machines to solve problems traditionally handled by human intelligence. Artificial intelligence includes 'machine learning' (ML). This special subgroup enables systems to learn through experience and, for example, adapt processes automatically and without programming, e.g. using special algorithms such as neural networks. A related term is 'deep learning' (DL), a subset of machine learning in which multi-layered neural networks learn from huge amounts of data.

Focus on AI-based video analysis

As AI is such a broad discipline, the following section will focus on how computer vision or AI-based video analysis[HB1] is used for classification and detection tasks in industrial applications. From reading data from remote monitoring and predictive maintenance, vehicle identification, agricultural drones and outdoor patrol robots to automatic optical inspection (AOI) of minute defects on golf balls [3] and other products, computer vision and video analytics enable higher productivity and efficiency for industrial applications.

Shifting AI to the edge of the IIoT

As already mentioned, the proliferation of IIoT systems generates huge amounts of data. For example, the many sensors and devices in a large oil refinery generate up to 1 TB of raw data per day [4]. Immediately sending all this raw data back to a public cloud or private server for storage or processing would require considerable resources in terms of bandwidth, availability and power consumption. For many industrial applications, especially highly distributed systems in remote regions, it is not feasible to constantly send large amounts of data to a central server.

With these reasons in mind, IIoT applications are therefore moving AI and ML capabilities to the edge level of the network, enabling greater pre-processing power directly on-site. More specifically, advances in the processing power of edge computers have meant that IIoT applications can now leverage the power of AI-powered decision making in remote locations. When field devices are connected to local edge computers with powerful processors and AI, not all data needs to be sent to the cloud for analysis. In fact, the proportion of data created and processed at the remote and near edge locations is expected to increase from 10% to 75% by 2025 [5], and the overall AI hardware market at edge level is expected to grow at a compound annual growth rate of 20.64% from 2019 to 2024 [6].

Choosing the right edge computer

If you want to incorporate artificial intelligence into your industrial IoT applications, you should consider a few aspects. Even though most of the work associated with training your AI models still takes place in the cloud, the trained inferencing models must ultimately be deployed in the field. AIoT edge computing essentially allows inferencing to be performed on the AI in the field, rather than sending raw data to the cloud for processing and analysis. For AI models and algorithms to run effectively, industrial AIoT applications depend on a reliable hardware platform for the environment. When choosing the right hardware platform for your AIoT application, the following factors should be considered

1. processing requirements for different phases of AI implementation,

2. edge computing levels,

3. development tools,

4. physical environmental conditions.

Processing requirements for different phases of AI implementation

The three phases in building AIoT applications.

© Moxa

In general, the processing requirements for AIoT computing depend on how much computing power is required and whether [HB1] an accelerator is needed in addition to the CPU. Since each of the following three phases of building an AI edge computing application uses different algorithms to perform different tasks, each phase has its own processing requirements (Figure 1).

Data acquisition: In this phase, large amounts of information are collected to train the AI model. However, unprocessed raw data alone can contain duplicates, errors and outliers. Pre-processing the collected data in the initial phase to identify patterns, outliers and missing information therefore allows errors and systematic biases to be corrected. The computer platforms typically used for data acquisition are usually based on Arm Cortex or Intel Atom/Core series processors. In general, the I/O and CPU specifications (and not the GPU) are more important for data acquisition tasks.

Training: AI models need to be trained on modern neural networks and resource-intensive machine learning or deep learning algorithms. These require more powerful processing capabilities, e.g. powerful GPUs, to support parallel processing in order to analyze large amounts of collected and pre-processed training data. Training an AI model involves selecting a machine learning model and training it using the collected and pre-processed data. During this process, the parameters must also be evaluated and adjusted accordingly to ensure accuracy. There are numerous training models and tools to choose from, including industry-standard deep learning development frameworks such as PyTorch, TensorFlow and Caffe. Training is usually conducted on dedicated AI training machines or cloud computing services such as AWS Deep Learning AMIs, Amazon SageMaker Autopilot, Google Cloud AI or Azure Machine Learning instead of on-site.

Inferencing: In the final phase, the trained AI model is implemented on the edge computer so that it can quickly and efficiently make conclusions and predictions based on newly collected and pre-processed data. Since the inferencing phase generally consumes less data processing resources than training, a CPU or light accelerator may be sufficient for the AIoT application in question. Nevertheless, a conversion tool is needed to convert the trained model so that it can be run on specialized edge processors/accelerators, such as Intel OpenVINO or NVIDIA CUDA. Inferencing also includes several different edge computing levels and requirements, which are discussed in the following section.

The edge computing levels

Although AI training is still mainly performed in the cloud or on local servers, data collection and inferencing necessarily take place at the edge of the network. Since inferencing is the phase in which the trained AI model does most of the work to achieve the application goals - i.e. making decisions or performing actions based on newly collected field data - it is also necessary to determine which of the following edge computing levels are required in order to select the appropriate processor.

Lower edge computing level

Transferring data between the edge and the cloud is not only expensive, but also time-consuming and leads to latency times. With low-level edge computing, only a small amount of user data is sent to the cloud, which reduces delay time, bandwidth, data transfer fees, energy requirements and hardware costs. An Arm-based platform without accelerators can be used on IIoT devices to collect and analyze data to quickly draw conclusions or make decisions.

Middle edge computing level

This inferencing layer can handle various IP camera streams for computer vision or video analytics with sufficient processing frame rates. Mid-level edge computing covers a wide range of data complexity based on the AI model and performance requirements of the use case, e.g. facial recognition for an office entrance system versus a large public surveillance network. Most industrial edge computing applications also need to consider aspects such as a limited power budget or a fanless design for heat dissipation. Thus, a more powerful CPU, an entry-level GPU or a VPU may be used at this level. For example, Intel Core i7 series CPUs provide an efficient computer vision solution with the OpenVINO toolkit and software-based AI/ML accelerators that can perform edge-level inferencing.

Upper edge computing level

High-level edge computing processes larger amounts of data for AI expert systems that work with more complex pattern recognition, for example for behavioral analysis in automatic video surveillance in public security systems that can detect security incidents or potentially threatening events. Inferencing at the upper edge computing level generally uses accelerators, including high-end GPUs, VPUs, TPUs or FPGAs, which require power of 200 W or higher and therefore generate heat. As the power consumption required and the heat generated can exceed the limits at the far edge of the network, e.g. on board a moving train, high-edge computing systems are often used in locations close to the edge, e.g. in a train station.

The development tools

Several tools are available for different hardware platforms to speed up the application development process or improve overall performance for AI algorithms and machine learning.

Deep learning frameworks

Consider using a deep learning framework, i.e. an interface, library or tool that makes it easier and faster to create deep learning models without having to deal with the details of their underlying algorithms. Deep learning frameworks provide a clear way to define models using a collection of pre-built and optimized components. The three most popular are:

- PyTorch was primarily developed by Facebook's AI research lab and is an open-source machine learning library based on the Torch library. PyTorch is free and open source software released under the modified BSD license and used for applications such as computer vision and natural language processing.

- TensorFlow enables rapid prototyping, research and production with TensorFlow's easy-to-use Keras-based APIs, which are used to define and train neural networks.

- Caffe features an expressive architecture that allows models and optimizations to be defined and configured without hard coding. In Caffe, a single flag can be set to train the model on a GPU machine and then implement it on commodity clusters or mobile devices.

Hardware-based accelerator toolkits

AI accelerator toolkits are offered by hardware manufacturers and are specifically designed to accelerate AI applications such as machine learning and computer vision on their platforms.

- The Intel OpenVINO (Open Visual Inference and Neural Network Optimization) toolkit from Intel is designed to help developers build robust computer vision applications on Intel platforms. OpenVINO also enables faster inferencing for deep learning models.

- The NVIDIA CUDA toolkit enables high-performance parallel computing for GPU-accelerated applications on embedded systems, data centers, cloud platforms and supercomputers based on NVIDIA's Compute Unified Device Architecture (CUDA).

Ambient conditions

Last but not least, the physical location where the application is to be implemented must also be taken into account. Industrial applications should have a wide operating temperature range and suitable mechanisms for heat dissipation. Certain applications also require industry-specific certifications or approvals. Many applications are subject to size constraints, so edge computers with a small form factor are preferred.

In addition, highly decentralized industrial applications in remote locations may require communication via cellular or Wi-Fi connectivity. An industrial-grade edge computer with integrated LTE cellular connectivity eliminates the need for an additional cellular gateway, saving valuable cabinet space and deployment costs. In addition, a redundant cellular connection with dual SIM support may be required to ensure data transmission even when the signal is weak.

Edge computing for industrial AIoT applications - the example of mechanical engineering

To see how real-world industrial applications can enable and benefit from AIoT edge computing, consider the following example.

Intelligent production lines - networked intelligent manufacturing

Networked, intelligent production line

© Moxa

Essentially, it is a digitized process that starts with intelligently integrated adjustment orders from the ERP, which are automatically converted into an intelligent purchasing and production plan and distributed to the intelligent machines of the mass and customer-specific production lines via an MES/SCADA system. This must happen smoothly and seamlessly: MES, WMS and AGV distribution systems orchestrate internal logistics to move parts, tools and products smoothly between the production lines and the warehouse. Finally, by connecting external supply chains with internal logistics, end-to-end production can be achieved.

Machine control and data integration

Intelligence is moving to the edge: Moxa supports companies in networking production lines.

© Moxa

The greatest challenges in mechanical engineering lie in the areas of machine control and data aggregation. Edge computers must be as compact as possible due to limited space and must be able to be mounted on the DIN rail. They also require a fast x86 CPU, various I/O interfaces, wireless support (WiFi/3G/LTE) and additional memory with mSATA or SSD support.

Moxa's edge computing solution MC-1220 is a very small device that features an energy and performance efficient 7th generation Core i7 CPU. A built-in system diagnostics engine with remote monitoring via SNMP allows production processes to be controlled and monitored from anywhere, and a wide range of interfaces connect all peripheral devices. The computer supports a wide operating temperature range from -40 to 70°C for hot and cold environments and is accessible from the front thanks to DIN rail mounting.

Fast processors and graphics processing units are required for quality control of the parts produced. The platform used must also be suitable for artificial intelligence. Once again, various I/O interfaces are required, as well as wireless support (via WiFi/ 3G/ LTE) and additional memory with mSATA or SSD support. With its fast CPU and GPU, the MC-1220 ensures that defective components are analyzed quickly and thoroughly and triggers an alarm so that defective parts can be removed from the product line. The corresponding data can be quickly processed, saved or transmitted in real time via WiFi or LTE.

Literature

[1] DalleMule, L. & Davenport, T.H. (2017). "What's Your Data Strategy?" Harvard Business Review, May-June 2017, 112-121.

[2] TechTarget (2019). "Artificial Intelligence of Things (AIoT)"

[3] Wu, H.H., Su, J.W., & Chen, C.L. (September 28, 2016) "Automatic optical inspection system design for golf ball." Proc. SPIE 9971, Applications of Digital Image Processing XXXIX, 99712S.

[4] Bechtold, B. (2018) Cisco. Beyond the Barrel: How Data and Analytics will become the new currency in Oil and Gas

[5] Van der Meulen, R. (2018) Gartner. What Edge Computing Means for Infrastructure and Operations Leaders

[6] Markets and Markets (2019). "Edge AI Hardware Market by Device (Smartphones, Cameras, Robots, Automobile, Smart Speakers, Wearables, and Smart Mirror), Processor (CPU, GPU, ASIC and Others), Power Consumption, Process, End User Industry, and Region - Global Forecast to 2024"

The authors

Ethan Chen is Product Manager at Moxa.

Alicia Wang is Product Manager at Moxa.

Angie Lee is Product Marketing Manager at Moxa.