Siemens Smart Infrastructure manufactures smoke detectors for fire protection in numerous variants and medium to large quantities on automatic systems. The components used are placed on the circuit board using automatic placement machines and then soldered from above. The EU-wide ban on solder alloys containing lead is forcing manufacturers to use lead-free solders, which, however, have poorer soldering properties. This results in increased reject and failure rates.
This makes reliable automatic quality control systems all the more important. In most cases, these are camera-based image processing solutions that use suitable software packages to make in-order/not-in-order (IO/NIO) classifications based on image analyses. However, their selectivity has not always been satisfactory. Particularly when used for critical safety functions, the test criteria must be trimmed towards the 'safe' side, as fire detectors must be extremely reliable. However, this results in increased reject rates with corresponding cost disadvantages.

To reduce these, the Swiss company Compar has set itself the goal of using additional artificial intelligence solutions in the form of self-learning neural networks for image analysis. In addition, such tasks at Siemens Smart Infrastructure are to be integrated into higher-level IT structures as part of Industry 4.0 concepts.

"The image processing specialist Cognex has developed ready-made software packages in the form of plug-in modules for such tasks under the name 'ViDi'," explains Lukas Vassalli, developer at Compar.

The integration of AI

As a hardware requirement, a powerful graphics processing unit (GPU) should be available on the computer used, at least in the training phase. An essential component of the software library is a neural network that is already partially pre-structured so that the user can quickly start training. This is necessary before the first use and is done by providing the neural network with a certain number of images as 'training material'. It can then evaluate new images independently according to the desired criteria. The wealth of knowledge acquired during training is constantly expanded and refined over the course of use. The application at Siemens Smart Infrastructure involves the assessment of solder connections and the detection of assembly errors.

The overall system

The image is split into a good and a bad part pattern. At the top, the error is detected with an error certainty of 0.99 (99 % NOK) and at the bottom with an error certainty of 0.02 (2 % NOK, i.e. 98 % IO).

© Compar

The complete system consists of a camera and an illumination station designed for the application, which holds the circuit boards, as well as an industrial PC with the 'Visionexpert' program. The system is supplemented by the 'ViDi' package, which works as a 'black box'. It analyzes the transferred images using its neural network and returns corresponding assessments. This is done instantaneously within milliseconds of the production line cycle.

Before the start, the system was pre-configured by Compar using images of sample parts provided. During operation, the system can be trained by the user with new products or retrained with variants of existing products as required. Thanks to the high computer performance, only a few minutes are required for such training phases. During training, the system can either be 'fed' directly with photos or errors can be highlighted in advance using color coding in supervisor mode. After a short training session, the user is able to carry out such tasks themselves. At Siemens Smart Infrastructure, around 50 images of good parts and the same number of bad parts were sufficient.

The 'ViDi' processes

Processes during testing: The camera and lighting units provide an image of the circuit board. The 'Visionexpert' software performs its part of the evaluation and at the same time initiates a parallel analysis by 'ViDi'. The results are then incorporated into the 'Visionexpert' evaluation. This results in control commands to the process PLC and messages to the company's higher-level IT structure.

© Compar

The 'ViDi' software consists of the three modules red, green and blue, of which the 'red' and 'blue' modules are used in the application described. The 'blue' module, known as the 'locator', checks the PCBs for correct placement. It identifies solder joints and component positions as well as imprints, whereby variances can be preset. ViDi 'red' then takes over the classification into IO or NIO parts. There is a choice of different approaches during training: for example, instead of the two categories IO/NIO, only IO parts can be specified. In this case, the AI will automatically classify everything that is not clearly recognizable as IO as NOK.

An important feature of the 'ViDi' analysis is the numerical evaluation of the classification of the respective result. Although the system always classifies reviewed images according to the criteria 'IO' or 'NOK', it always outputs a percentage confidence value. This indicates the percentage of confidence the software has in its judgment.

Selectivity as a reliability feature

The scale ranges from 0 (=100 % IO) to 1 (=0 % IO or 100 % NOK). The frequency distribution of these classifications is displayed statistically in the form of diagrams with, for example, green color for IO and red color for NOK results. They take the form of two bar charts in green and red respectively, which may partially overlap.

A simpler representation is obtained by plotting the cumulative scattering ranges normalized to one. Depending on the task and evaluation criteria, these can either partially overlap or form two clearly separate groups. If the training has run optimally, there is no overlap between the cumulative frequency ranges, which demonstrates the good discriminatory power of the method. If this is not the case, you end up in the decision area between 'false-positive' and 'false-negative' classifications. In such cases, the optimal determination of the so-called Treshhold value plays an important role. If this is placed more on the safe side, for example, the risk of failure of safety-relevant components at the customer is minimized. The reverse strategy, on the other hand, can reduce the internal scrap rate if necessary.

Interaction with 'Visionexpert'

Klaus Vollrath is a freelance journalist, photographer and cineaste at the editorial office Klaus Vollrath in Aarwangen/Switzerland.

© Compar

Of particular interest to users is the integration of the 'ViDi' options with the 'Visionexpert' image processing software developed by Compar. As the main component, the software initially takes over the external hardware handling, i.e. the connection of the numerous possible camera models and other peripherals. Another task is image data management and the transfer of image data to be analyzed to 'ViDi'. The returned results are used internally, visualized and finally integrated into the decision-making process.

Despite all the automation, humans always retain the decision-making power by specifying inspection criteria and decision guidelines such as the threshold level.

In addition to the results of the 'ViDi' examination, Visionexpert's own capabilities are used to analyze and assess a test specimen. In contrast to the 'ViDi' plug-in, the software can, for example, measure dimensions down to the micrometer range with high accuracy and make decisions based on the results. And last but not least, 'Visionexpert' takes over communication with the company's higher-level IT.