Customer requirements in mechanical and plant engineering are constantly growing: ever shorter production and changeover times, permanent deadline pressure, a growing number of variants and so on. In order to meet these challenges, digital twins are increasingly being used before commissioning. In addition to methodical test planning and execution, they enable any software errors to be rectified at an early stage. At the same time, however, the system architecture of the control software in modern plants is becoming increasingly heterogeneous, not least due to machine operators who - thanks to technologies such as smartphones - are increasingly open to intuitive operating concepts and customizable interfaces in their plants.

The system architecture often consists of the core components HMI, PLC and NC kernel. As there are different interfaces between these components and the system and control level in the company, the complexity of such systems continues to increase. Testing is therefore essential in order to check whether control software actually works as intended. However, the most important part of the test activities - namely the evaluation of the interaction of the control software - still often only takes place during integration and commissioning.

Basic structure of a test station for automated testing of control software.

© ISG Industrial Control Technology

So much for the status quo. In the age of digitalization, many companies would now like to be able to start the test phase earlier. To make this possible, more and more companies are turning to virtual commissioning (VIBN). This involves connecting the real control hardware and software with a virtual model of the system. In the latter, the behavior can be simulated in real time, creating a so-called hardware-in-the-loop (HiL) setup. This means that the entire system can be subjected to both a good and a bad case test at a much earlier stage. While the good-case tests aim to check the performance and functionality required by the customer, the bad-case tests test the reliability and robustness of the software by checking the reaction of the control software in explicit hazardous and fault situations. Essential acceptance tests on the real system at the manufacturer's premises (factory acceptance test) or at the customer's premises (site acceptance test) can thus be completed much more quickly.

To date, however, VIBN tests have presented companies with certain challenges. In principle, they usually follow a similar pattern: an inspector carries out the test steps while manually performing all the specified operating actions on the HiL setup using an extensive checklist. He provokes fault situations by making appropriate interventions on the digital twin. This allows the effect of the control algorithms for error detection and primary error reactions to be assessed.

The problem with performing these tests manually is that errors can always creep in when it comes to observing the states of the system - such as the incorrect interpretation of a test step or inattention on the part of the tester. In addition, when changes are made to the control software or system components, operating actions often have to be carried out from scratch (regression tests) - which takes a lot of time.

It is also unavoidable that the modularity and configurability of the systems result in high repetition rates for individual tests. Last but not least, continuous updates from the control system manufacturer also ensure that regular regression tests of the entire control system software are necessary.

In order to master these challenges, a suitable test automation tool (TAW) is required.

The criteria of automated testing tools

Excerpt from a test sequence. In this example, an axis is moved up to the software limit switch.

© ISG Industrial Control Technology

Various aspects need to be considered when selecting a suitable solution: Firstly, the TAW should support the definition of reusable test modules. This is necessary in order to be able to reuse the created test sequences in different phases of the development of a system. It is also essential that test modules can be created without programming knowledge. This is the only way to ensure that all project participants can also take part in the test phase.

It would also be helpful if the TAW presented the test sequences in an easily understandable way and was intuitive to use. This also applies to the interaction of the TAW with the control system: no expert knowledge of communication protocols or similar should be required for this. As the TAW is usually used to test very complex systems, it is also advantageous if the tool allows parallel processes, branches and loops in the test sequence.

Several control units working together

When it comes to test automation, companies are often faced with the challenge that complex systems contain several controllers. The TAW should therefore be able to test different digital twins and their controllers in interaction. Furthermore, a detailed logging function should not be neglected: After all, the purpose of test automation is to quickly and reliably localize the causes of errors so that they can be easily rectified. This also applies to the recording of input and output data of the individual test modules as well as the test reports.

In the course of ever shorter production times, the time component is crucial in test automation: in order to achieve the shortest possible turnaround times, it should be possible to make and check changes directly during the test run. Another important aspect is the debugging of test modules: Breakpoints should be set and tests should be able to be processed and followed step by step.

Last but not least, the specific structure of the systems plays a role in test automation that should not be underestimated. As a rule, systems have a modular structure and therefore have a high degree of variance with regard to the respective customer-specific design. Depending on the system structure, the hardware and software addresses of components, parameters, inputs and outputs etc. can change. The TAW should be prepared for this. It is necessary to address parameters symbolically in order to be able to adapt test modules easily, safely and quickly to the respective system design. Another challenge is the control system, which can be freely selected by the customer. This is why a TAW with the greatest possible manufacturer-independent compatibility with control systems and components is required. Ultimately, the aim is to use the same test modules for testing completely independently of the manufacturer of the respective control system.

Digital twin of a control panel for operating machine components. The operator can use the buttons to trigger the actions of the machine components.

© ISG Industrial Control Technology

If companies ultimately opt for test automation, they benefit in many ways: unmanned tests can easily be carried out at night or at the weekend. Accordingly, continuous tests are no problem, so that companies can detect even sporadic errors. Fire department operations' on site are also no longer necessary, as errors in the control software can be eliminated in advance, which in turn creates a long-term enhanced quality image for the end customer.

One example of a modern test automation tool, as described above, is the 'ISG-dirigent' tool. It was developed as part of a joint research project between the Institute for Control Engineering of Machine Tools and Production Equipment (ISW) at the University of Stuttgart and the company Industrielle Steuerungstechnik GmbH (ISG). The software is based on two components: the simulation solution ISG-virtuos, which is able to simulate digital twins, and expecco, a framework for test automation from the company Exept AG.

TAW using a concrete example

From the list of loaded libraries that are available for the test sequence, the user selects the blocks that he wants to use for his test sequence.

© ISG Industrial Control Technology

A control system frequently used in the machine tool environment is the Sinumerik 840D sl from Siemens, for which the structure of ISG-dirigent is described below: In addition to the standard library, the basic tool expecco already includes other important libraries that are necessary for testing such systems. For example, the Qt library enables the communication of the test blocks with the elements of the user interface of the CNC control, while the VNC library allows the transmission of keyboard entries to the control panel of the Sinumerik. The SCP library, in turn, enables the transfer of files (such as NC programs) from and to the controller.

But only the ISG library for the Siemens Sinumerik 840D sl complements the tool to the full functionality of ISG-dirigent. Based on the 'OPC UA Client Interface' and 'ISG-virtuos Client Interface' libraries for communication with the controller and the digital twin respectively, this library is divided into the following sub-areas:

• Blocks of the control panel group (OP 012) for user-friendly interaction of the test blocks with the controller in relation to the control panel.
• Blocks of the Machine Control Panel group (MCP483) for the interaction of the test blocks with the controller in relation to the machine control panel.
• Blocks of the Machine and Settings Data group for access by the test blocks to this control system data.
• Blocks of the Arithmetic Parameters and User Data groups for access by the test blocks to this control system data.
• Blocks of the PLC Data group for the access of the test blocks to this control system data.

If the system manufacturer has systematized its test sequences, the most efficient way to start test automation is during virtual commissioning. However, this also includes defining the test sequences on the basis of individual test steps, recording the expected results of a test sequence and creating protocol templates. Plant manufacturers can then start to identify the common parts between test sequences. In this way, a library of reusable test steps can be built up step by step. The protocol template in the TAW provides a quicker overview of how a test went and where errors may have occurred.

ISW is also conducting research into the automation of commissioning tests. This includes the intelligent generation of data for the test sequence and the expansion of test execution into the real-time part of the HiL setup. It is planned to add further libraries for controllers from Fanuc, Heidenhain, Bosch-Rexroth and Beckhoff as part of the work on ISG-dirigent.

Authors:
Dr.-Ing. Gerhard Krebser is responsible for the test automation of control software at ISG Industrielle Steuerungstechnik;
Karl Kübler is a research assistant at the ISW at the University of Stuttgart.