Machine learning is one of the big buzzwords in the field of industrial manufacturing. But how can the technology be implemented in practice in a meaningful way and what specific challenges arise? Representatives from industry and research addressed these questions at the VDI conference 'Machine Learning in Production' in Baden-Baden at the end of last year.

In the search for a suitable implementation strategy, three current methods emerged that differ in terms of their degree of autonomy: Is the goal for the ML solution to function without user interaction - i.e. autonomously? Or is the goal interaction with a data scientist? Secondly, the location of the calculation plays a key role: should the data analysis take place at the edge, i.e. in the real-time capable automation devices? Or is all the data stored in the cloud, where the analysis takes place with a lot of computing power?

Below are the simplified methods in detail with their respective advantages and disadvantages.

Method 1: Interactive data analysis

This approach corresponds to the classic procedure for data analysis: an expert manually reviews the data and develops an understanding of the analysis problem. Based on this information, a selection of analysis methods is made. The expert applies these methods to the data and manually optimizes the parameters of the method. Finally, the results are graphically processed and presented.

A typical feature of this approach is that the expert usually has a background in computer science or data science and uses special data analysis tools for the analysis. The methods are selected and applied on the basis of data properties such as time dynamics, dependencies and noise. The comparison with domain knowledge, on the other hand, requires a strong interdisciplinary dialog within the company. Such an analysis therefore usually takes several days to weeks. This method is likely to be the normal state of affairs in most cases. In other words, it corresponds to the procedure that data scientists learn and that many IT companies in this field have used successfully in the past. As humans are the slowest component in this method, server systems or good PC platforms are usually used for the analysis. The data can easily be stored in the cloud or in database systems.

An example of this approach: Together with Miele, the Fraunhofer Institute for Industrial Automation at Fraunhofer IOSB carried out an interactive data analysis. The aim was to investigate the cycle time of a dishwasher production line at the Bielefeld site. Miele's process experts provided the data, which the Fraunhofer Institute's data analyst then processed manually.

In the course of this preparation, hybrid machines were used as a method to map the typical behavior of the plant in terms of cycle time in a first step. Once this data had been processed graphically, both the process experts and the data analysts met for an evaluation. Times were identified in which the machine operated outside of the expected value.

Figure 2: Illustration of the use of a step sequence automaton to identify temporal variances in the production sequence. In-depth analyses of the state transitions provide important information for explaining the variances.

© Fraunhofer IOSB-INA

Figure 2 shows the simplified step sequence automaton. This automaton was used to find the state sequences for whose transition the greatest variances in production time occurred. These production time variances were then compared with failures and other production-related delays. Finally, only unexplained cycle time deviations remained that could be investigated further. To this end, dimension reduction procedures, graphical analyses and methods for detecting outliers were combined and the results were then discussed with the experts. At the end of this process, the process experts drew up hypotheses for the deviations in the observed cycles based on the data analysis and derived corresponding actions.

Advantages:
The main advantage of interactive data analysis is that this procedure has been established and proven for a long time. Human experts are good at selecting procedures and parameters based on the data characteristics. They are also able to interpret the results and communicate them to their clients in their own language. The tools are also usually tailored to this approach. Another advantage is that standard software can be used for the analysis.

Disadvantages:
The disadvantage of this approach is that the expert not only needs a deep understanding of the analysis methods, but must also have domain knowledge. In practice, this approach usually fails due to the availability of experts; SMEs in particular are not in a position to hire them on the labor market. Another disadvantage is the time required for this, which makes this method less useful for time-critical issues.

Method 2: AutoML

The complexity of the 'ML pipeline' - from data acquisition, the selection and creation of suitable features and models, hyperparameter optimization and analysis of the results obtained, through to model monitoring - is difficult for a non-ML expert to master. One possible approach is AutoML, i.e. software support for method selection and method parameterization. AutoML tools usually analyze the data in order to automate the method configuration. Furthermore, different methods and parameters are often tried out and the best results are adopted.

The Machine Learning working group at the Institute of Computer Science at the University of Freiburg, for example, is working in this field. The Freiburg scientists' special topic is automated hyperparameter optimization. AutoML takes over and automates the often tedious and difficult manual setting of hyperparameters. In addition, AutoML enables greater reproducibility, which is also important with regard to the ethical AI guidelines published by the European Commission. The aim of these guidelines is not only to achieve legal and ethical results, but also reproducible results.

AutoML implementations for typical ML software tools such as WEKA and Skikit-learn or their extensions (Auto-WEKA/Auto-Sklearn) aim to support non-expert users of ML techniques by assisting users in the selection and parameterization of procedures. In detailed investigations by the Freiburg scientists, Auto-WEKA was the procedure with the lowest error rate in 15 out of 21 data sets. In three data sets, the performance improvement of Auto-WEKA over the other methods was significant at 16%.

Advantages:
AutoML will enable domain experts to automate the interactive approach or guide the user through the analysis process. In most cases, existing, established tools can continue to be used so that companies can implement migration scenarios. Furthermore, this method supports the training process in companies and helps non-specialist employees to familiarize themselves with the topic of ML.

Disadvantages:
In practice, these approaches simplify the use of ML methods; in particular, the parameterization of the methods is accelerated. However, the selection of methods and, above all, the recognition of incorrect results still require an expert. Although these approaches have great potential, further research is still required in this area. In particular, the relationship between data characteristics and the ML method used is still poorly understood. Another open point is the integration of domain knowledge into the data analysis process, for which generic, easy-to-use approaches are lacking.

Method 3: Generic ML procedures

A third, completely different approach, which was discussed intensively at the VDI conference, is to pack a generic data analysis method directly into the edge and already perform many machine learning tasks there. The appeal of this approach lies precisely in the fact that efficient implementations for distributed automation systems can be created in this way - while fulfilling important properties such as reliability and real-time capability.

For example: Neural networks are powerful ML tools that can be used to generate very precise data models, especially in industrial environments - always assuming that sufficient time series data of sufficient quality is available. In this way, certain tasks can be automated by software agents - in particular the optimization of certain target variables in factory processes (quantity, time, quality and costs), but also the monitoring or preview of process and status variables (plant behaviour, failures or malfunctions).

Figure 3: Automation technology and sensors in the chemical plant provide the input variables. This is used to train a neural network. Now a desired operating state (e.g. maximizing the production quantity) and the required setting variables can be determined automatically. The image shows the output variables in the non-optimized case (measured values) versus the optimized case (calculated values).

© Ahorner

For this purpose, sensor data from the process control system or from the factory's decentralized control systems is merged into an empirical plant model: This includes target variables such as KPIs, disturbance variables, non-observable signals and adjustable or controllable variables. In practice, data pre-processing still accounts for up to 80% of the human work involved in creating the data model. The procedure is as follows: The neural network is first trained with a portion of historical data (offline learning). The model independently forms a formula to calculate the historical output results from the given input data. Another part of the historical data is then used to check how well the neural network can now independently calculate further output results from previously unknown input data. In this way, the model is validated and tested. Finally, the data model is connected to the system in order to adapt the algorithm independently and continuously using current operating data (online learning). In the offline phase, historical data is used; in the online phase, the model is implemented and thus connected to the actual data world. Figure 3 shows an example of a neural network forecasting the production volume of a chemical factory.

Advantages:
The advantage of this approach is that the existing multitude of ML methods is replaced by a single method. This also simplifies the parameterization of the methods; parameter settings can be automatically selected for a single method based on the data characteristics and the task, analogous to the AutoML approach. Generic implementations can also be made available on devices, making 'on-the-edge' implementation more likely.

Disadvantages:
The disadvantage of this approach is that no method is currently recognized as generically applicable for the different data types and the different tasks. Deep neural networks are certainly the most promising candidate, but have disadvantages in terms of data volume requirements, resilience of the results and applicability for time-dynamic systems. Various research gaps still need to be closed in this respect.

Summary

In summary, it can be said that None of the three approaches outlined currently solves all the challenges in the context of ML for production. The most viable approach at the moment is to focus on interactive data analysis without losing sight of other approaches. However, companies need to clarify how they intend to recruit, train and retain the necessary number of data scientists. They also need a concept for how these experts can cooperate with the company's domain experts in the long term. This is particularly difficult for SMEs. On the other hand, the other two approaches are currently still 'work-in-progress', so companies should plan to minimize risk, for example in the form of research projects, before commercialization. Switching between the approaches is not ruled out in principle, but usually entails delays and additional costs.

Authors:
Markus Ahorner is Managing Director of Ahorner & Innovators in Ratingen;
Jens Eickmeyer is an ML expert at Fraunhofer IOSB-INA in Lemgo;
Dr. Oliver Niggemann is a professor at the Institute for Automation Technology at the University of the Federal Armed Forces in Hamburg;
Peter Seeberg is the owner of the consulting firm Asimovero.AI.