Interactions within the cyber-physical test network CPPrN with its elements cyber-physical test system (CPPrS) and cyber-physical test object (CPPrO) as well as the embedding in its production environment (CPPS).

© Fraunhofer IFF

The key theme of 'Industry 4.0' aims to develop and expand cyber-physical systems (CPS) for production in order to network machines, storage systems, equipment and product statuses in companies horizontally and vertically along the value chain and to exchange the associated information (big data), initiate process steps and control and regulate them independently. The 'smart factory' model leads to production logistics in which the products carry their production history with them, know their status and independently determine the production routes.

Industrial companies that develop, manufacture and sell sensors and systems for non-destructive component and part testing must embrace this trend, as their technologies will become an essential part of controlling or even regulating production and the quality of the finished product.

Cyber-physical testing systems enable interaction between testing and production. To achieve this, inspection systems need to be enhanced with features such as intelligence, flexibility and adaptability. In this way, the inspection system, as developed by the Fraunhofer Institute IFF in Magdeburg, can adapt and autonomously adjust to changing requirements on the basis of integrated model knowledge - changes in terms of product properties or the inspection environment, for example.

Convertible testing system

The combination of virtual and real test systems based on the component data leads to model-based testing. This is followed by multimodal data evaluation. Based on this, the component status is updated in the CPPrO.

© Fraunhofer IFF

This means that the test system must be highly flexible in order to cover the widest possible range of requirements, for example different products or variants. And the test system must be adaptable in the sense that it can be changed or converted in a single design step. To this end, all objects - i.e. product, process and test system - have a digital image, the model. This model enables the interdisciplinary linking of virtually simulated and real measured test data for the design, optimization, process planning and execution of tests at a high semantic level.

The test system is networked with the cyber-physical test network (CPPrN) to ensure that all relevant information is available to all objects involved at all times. Described in abstract terms, it looks like this: Test systems with the aforementioned properties are called cyber-physical test modalities (CPPrM). They are specifically assembled as autonomous function modules in combination with cyber-physical kinematics modules CPPrK (manipulation units, test robots) to form the cyber-physical test system (CPPrS). They work in the Cyber-Physical Production System (CPPS) and receive input information from the product file of the Cyber-Physical Test Object (CPPrO) about the target product condition as well as information recorded in previous tests about the current product condition. On the other hand, they pass on newly recorded information about the current product status to the test object via the network.

This testing methodology enables two new properties of the cyber-physical testing system to be achieved:

1. The information from product development, production planning as well as the production process is used to design the inspection processes - the inspection is therefore production-controlled.
2. The test results (product attribute 'quality') control production (i.e. the process in the subsequent production step or test step) - production is therefore quality-controlled.

The complete determination and control of product quality requires the networking of all test modalities including the test data within the cyber-physical test network. It enables the flexible use of complementary test modalities as so-called multimodal testing with different test procedures and evaluation methods.

The aim of this testing methodology is to guarantee consistently high product and process quality with the help of a model-based linking of virtual and real testing functionality as well as subsequent comprehensive data fusion and data analysis to complete the product file of the cyber-physical test object. By taking a stringent view of the entire value chain, the testing processes therefore become an integral part of the production and manufacturing processes. The resulting paradigm shift means that testing is not a one-off defined or programmed functionality, but reacts adaptively and flexibly to changing requirements.

Similarly, testing is no longer a 'necessary evil' for checking the quality achieved; rather, multimodal testing controls or regulates the production process in the individually planned, optimum quality. In this concept, the measures for quality assurance through testing are an elementary component and basis for economical and quality-assured production.

What does a cyber-physical inspection system consist of in detail?

Model-based measuring and testing

As a rule, conventional measuring and inspection systems for process control and quality inspection work independently and are designed for a defined and limited measuring or inspection scope. Changes to the product to be inspected or to the inspection task require intervention and the adaptation of configurations, inspection plans and target specifications, which results in production interruptions and expenses in work preparation.

Model data and simulations are key to equipping future measuring and testing systems with the necessary flexibility and adaptability - be it 3D geometry information of the test object or model information of the test arrangement (geometric and functional description). Ensuring flexibility even with changing product requirements requires the measurement and inspection systems to be networked with the overall production system.

Specific applications of the model-based approach are, for example, dimensional geometry, optical assembly, surface or defect testing based on geometry information of the component.

Model-based test planning

An inspection plan generally specifies where on the test specimen which characteristics are to be inspected using which determination method. A flexible system should be able to use an abstract description (for example: check whether the hole at position X has the correct diameter) to implement the steps required to determine the result itself and as automatically as possible. This is possible with the help of model-based inspection planning: the digital geometry model of the test specimen, the model for describing the geometric arrangement of the sensory elements (such as camera and lighting) and the functional description of the sensory data acquisition form the basis for a virtual measurement by simulation. The result of this measurement simulation is synthetic measurement data. These can be used in test planning for the automated determination of a suitable or optimum sensor position. By varying the spatial position of the virtual sensors and simulating the measurement, a series of synthetic measurement data sets is created. Their evaluation with regard to the completeness and quality of the measurement data enables the calculation of an optimum inspection perspective for determining the desired inspection characteristic. This eliminates the need to teach in new inspection positions.

If the inspection system has kinematics that move the inspecting sensor to successive inspection positions, the model information can be used for collision-free path planning of the kinematic system. If the scan paths and trajectories for the relative movement of the sensor and test object can also be simulated based on the model, the sensor system (test modality) and kinematics can be flexibly linked to the test system. Automatic and highly flexible inspection planning becomes possible.

Synthetically generated target states

The basis of every test is the description of a desired target state. The comparison of the existing, measured actual state and the desired target state provides the test result. Systems with flexible testing tasks therefore require a generic approach to provide the target information that describes a fault-free product condition. The use of model data and measurement simulations makes this possible. In addition, product condition data recorded in previous test steps - for example, permissible geometric deviations from the idealized 3D CAD product model - must be made available for subsequent test steps. This means that the target/actual comparison can be carried out more flexibly - especially where target states allow a certain variability in the actual states, such as with permitted component tolerances in production.

Multimodal data analysis through data fusion

In the narrower sense of optical inspection with image processing, data fusion or sensor fusion refers to the combination of data from several different or similar sensors in order to achieve more accurate, more reliable or more complex results. The fused information and data obtained in this way, which characterize the component, allow a comprehensive assessment of the component's condition and form the basis for the automatic adaptation of further processing steps and inspections.

A look at the assembly test

A sensor head mounted on a robot with image sensors and 3D measuring sensors determines the real measurement data during aircraft construction. The inspection system compares this with the synthetic data and marks faulty components in red.

© Fraunhofer IFF

The technologies developed by the Fraunhofer Institute IFF in Magdeburg have been tested in practice in the first pilot systems. A widespread inspection task in industry is inspecting the presence and completeness of assembly components, for example in aircraft construction. In this specific case, a robot-based inspection system automatically inspects all assembled add-on parts and joints on aircraft fuselage shells. The inspection system was developed in collaboration with the company Premium Aerotec.

The system takes the necessary information from the available 3D CAD data for the fuselage shell. The data shows exactly where each component must be located. The system uses this data to create synthetic measurement data in the form of synthetic images and 3D point clouds of the inspection features, so that each rivet location and each individual attachment part is represented exactly. The real measurement data is provided by a specially developed sensor head equipped with image sensors and 3D measuring sensors. Mounted on a robot, the sensor head scans each of the 1000 to 5000 positions and generates measurement data on the assembly status of the real add-on parts. The system compares this real measurement data with the synthetic data. If the two sets of measurement data match - i.e. if the components depicted on them are correctly assembled - the system marks the components in green as faultless. If it finds discrepancies, they are marked red as faulty, and yellow if they are unclear. In an inspection log, which can be operated interactively in a similar way to an app, the worker can display various evaluations - for example, all relevant drill holes or all parts that have been marked yellow and red. The system not only provides the operators with the photos of the components, but also the coordinates so that they can quickly find the component to be inspected - which is urgently needed, as an aircraft fuselage shell can be up to 11 meters long.

Authors:
Dr. Dirk Berndt is Head of Department and Head of the Measurement and Testing Technology Business Unit at the Fraunhofer Institute for Factory Operation and Automation IFF in Magdeburg;
Dr. Ronald Rösch is Head of Strategic Image Processing Research at the Fraunhofer Institute for Industrial Mathematics ITWM in Kaiserslautern;
Prof. Bernd Valeske heads the Department of Component and Part Testing at the Fraunhofer Institute for Nondestructive Testing IZFP in Saarbrücken.