Why is it so important to keep an eye on the workflow as a whole when working with artificial intelligence (AI)?

Johanna Pingel: Engineers working with artificial intelligence often focus primarily on the AI model. However, the workflow as a whole determines whether you are successful with AI. AI models need labeled data for training; it is therefore extremely important that the workflow also includes the cleansing, preparation and labeling of the data.

As Product Manager at Mathworks, Johanna Pingel works with artificial intelligence and has specialized in deep learning and computer vision.

© MathWorks

An AI model also offers added value when it is used in production systems or integrated into an embedded system. However, the AI model alone is usually not the complete system; rather, it is integrated into a larger system and must work perfectly with it. It is therefore helpful to understand the overall context and to know how all components interact with each other. For the development of AI-supported systems, simulations, tests and deployment must be understood as part of the workflow. It is important to bear in mind that deployment can take place on edge or embedded devices, for example, but also on enterprise systems and in the cloud.

Keeping all steps in mind right from the start is therefore a great advantage. If, for example, requirements are unexpectedly changed later in the development process, it is much more difficult to change direction.

What are the main aspects to consider in the individual steps of the workflow? How do engineers manage to build trust in a model, for example?

Pingel: The workflow of an AI development consists of four steps. Focusing on all four steps helps engineers to develop a complete, AI-supported system.

The first step is data preparation: AI models need large amounts of data in order to learn and make predictions. This data must be cleansed and available in a format that the model can work with. Importing and preparing all the data in such a way that a highly accurate model is obtained is therefore often time-consuming. The most important thing in this step is to use apps for data labeling, which significantly accelerate this time-consuming pre-processing - including of signal and image data.

The second step is modeling the AI: The heart of an AI is the model and there is a large selection of model types. Software such as Matlab offers the option of importing and testing different models. This makes it possible to determine which model is best suited to a particular application. Modelling apps enable automated training and supporting visualizations. The available models include those for machine learning, deep learning and reinforcement learning.

Simulations and tests are in third place. If an AI system is to be successful, it must work correctly in all conceivable scenarios. Having simulated all these scenarios and tested all use cases is an important milestone to ensure that you have built a system that works accurately and is ready for the final phase.

And last but not least, deployment: as AI spreads into more and more fields of application, the requirements for
requirements for deployment are becoming more complex. It is therefore important that deployment can take place on a wide variety of systems: from edge systems and the cloud to GPUs and MCUs.

A fully functional system is achieved when all use cases that are likely to be encountered by the model have been successfully simulated, tested and verified. Tools such as Simulink help to verify whether a model works as desired under all expected use cases. This avoids having to rework a model from scratch at great expense.

What are the typical challenges that engineers face when working with AI across the workflow?

Pingel: AI has its own unique challenges. Typical questions are what can I do if I don't have enough data, which model is suitable for which application and which platform can help provide the model.

AI needs large amounts of data to generate successful models. Matlab, for example, can be used to increase data volumes through simulations. In predictive maintenance applications, for example, it can be a problem to collect data on rarely occurring events or faults that would destroy the machine. This makes it difficult to train an AI with sufficient data on these errors. Tools such as Simulink and Simscape, which is used to design and simulate physical systems, can be used to create realistic pump models, for example. By simulating a wide variety of error scenarios, the data obtained can be used to train the AI model.

There is also a whole range of pre-trained models in the software tools that can be trained in a short time so that they are suitable for a new, individual solution. Engineers can therefore flexibly try out different solutions and determine which is best suited to their application.

My recommendation when choosing the right platform is to focus on the system requirements and not on a specific platform. It is important that the model is flexible and can be exported and run on different systems. You also need to be confident that you can generate the program code for the respective platform without introducing errors.

However, the most frequently asked question is: Where do I start? I always recommend learning the basic core concepts first and then focusing on the application in question. Start with reference examples to determine how AI fits into a particular application. Most importantly, realize that engineers don't need to be data scientists to develop deep learning applications. They already have the fundamental knowledge of their project and the data needed to succeed. To make it easier to get started, we also offer free tutorials for deep learning, machine learning and other more specific topics.

Best practices for engineers

What best practices should engineers follow to keep the workflow as a whole in mind when integrating AI into applications and projects? For example, how does an engineer know whether machine learning or deep learning would make more sense?

Pingel: Just be an engineer and think like an engineer: you don't have to be a data scientist to use deep learning and machine learning successfully. Engineers already have the skills to work successfully with AI.

Furthermore, use domain-specific tools that allow you to work quickly and interpret your data. Tools with functions that have been specially developed for signal processing, wireless applications, image data processing or text applications, for example. Use apps that do the manual programming for you. In addition to labeling apps that speed up the pre-processing of input data, you can quickly create, train and test AI models with apps such as Classification Learner and Deep Network Designer. Start with pre-built models built by experts from the AI community so you don't have to start from scratch.

As Product Manager at Mathworks, Johanna Pingel deals with artificial intelligence and has specialized in deep learning and computer vision.

© MathWorks

Machine learning and deep learning are the core or key technologies at the heart of AI. Although there are a number of helpful tips on what to consider when choosing between machine learning or deep learning, if you have large amounts of data but limited experience in choosing which features best represent that data, deep learning is the way to go as it automatically learns the most relevant features of the data. If your datasets are smaller, you are doing feature engineering and therefore want to select the best features for your algorithm, you can focus on machine learning and conventional classification methods such as support vector machine or decision trees instead.

Machine learning methods are by their nature usually easier to 'explain#, so many engineers opt for these methods because they want to know exactly how the model arrived at a decision. Deep learning can deliver extremely precise, but less intuitive "black box" results.

When integrating AI, you should ideally have access to both machine learning and deep learning methods. This allows you to try out which is best suited to your own application.

Which tools support engineers throughout the entire workflow, from prototyping to production?

Pingel: Engineers shouldn't spend unnecessary time on programming when they want to focus on more important things like sourcing data, developing models and testing accuracy. Apps can help to quickly prepare thousands of sample data or build model architectures with point-and-click tools. It is also important to build and test models so that they work in conjunction with all other components of the system. Simulation tools such as Simulink help to connect the individual parts and ensure that the AI model does what it is supposed to do in a system.

Another point is interoperability. When building models, it helps to benefit from the expertise of the wider AI community. To create an accurate and robust model, it is crucial to be able to import and export existing models from a wide variety of sources, regardless of platform.

What should be considered when using AI in security applications?

Pingel : There is growing interest in using AI models in safety-relevant applications with internal or external regulatory requirements. Even if different requirements apply to each industry, it is always important to provide evidence of training aimed at robustness as well as fairness and credibility. Safety is the core concern in a number of interesting areas:

There is a large area of ongoing research in verification and validation that seeks to take explainability further than simple confidence and proof that a model works under certain conditions. The focus there is also on models that are used in safety-critical applications with prescribed minimum standards. Sectors such as the automotive, aviation and aerospace industries themselves define what safety certification of AI should look like for their specific applications. Conventional concepts that are replaced or supplemented by AI must meet the same standards. They will only be successful if the outputs of the models are verifiable and show interpretable results.

Another important factor of 'Explainable AI' is that the output of the system must be identical to a person's expectations. This is something that engineers need to think about from the start: How do I communicate my results to the end user? Transparency is the key word here.

Robotics applications are ideal for the use of artificial intelligence. One of the megatrends is collaboration between humans and robots. In this application example, how is the human factor taken into account during the development of AI algorithms?

Pingel: If you look at the AI workflow, particularly in terms of collaboration between humans and robots, there are two factors to mention:

A relevant trend in the field of AI for this is explainability, or the ability to make people working with AI understand how machine learning models arrive at predictions. While explainability is necessary for all AI applications, it is essential for the interaction between humans and robots. Humans must be able to understand why a robot makes certain decisions. This makes interaction easier.

The second factor concerns human safety: when humans are in close proximity to robots, it is essential that safety comes before anything else. In many areas of robotics, a wide variety of data sources such as radar or lidar can be used alongside conventional RGB cameras to enable machines to detect the presence of humans in all conditions and regardless of lighting conditions. This makes it possible to ensure that it is safe for people to stay at a certain distance from robots.