Until now, industrial image processing systems were generally separate from the control technology. Vision experts solved the image processing tasks at hand using special tools in specific programming languages, usually on a separate PC or in an intelligent camera. In addition to high engineering and support costs, such systems have the disadvantage that time synchronization between control technology and image processing is virtually impossible. Beckhoff offers users an alternative to this and allows complete integration of the vision system into the controller, both in engineering and in runtime.

The company is thus also pursuing the path of concentrating intelligence in an Industrial PC for image processing. The camera on site therefore no longer requires its own intelligence, which enables more cost-effective systems. The solution in the control PC in software is very flexible and only limited by the computing power of the IPC. In addition, unlike intelligent cameras, all image data and intermediate results are available for an HMI or for saving images for further analysis.

Efficient engineering

The Beckhoff solution is based on the 'Twincat 3' automation software, which enables the engineering of elements such as control and drive technology, safety and now also vision tasks with the help of 'Microsoft Visual Studio'. This provides automation engineers with the same tool that programmers use for standard IT applications. A major advantage of this tool is its worldwide familiarity and acceptance. The interfaces to source code control, databases and test tools can also be used directly. In addition, the configurations for the various fieldbuses supported by 'Twincat' and, with 'Twin-safe', the configuration and programming of safety-related applications can be carried out in the same tool. Motion with NC PTP is also available, as is the configuration and programming of CNC applications. The decision to integrate image processing into the control system was therefore a logical one.

In the style of typical PLC programming, functions for contour and color recognition, keypoint features and measuring tasks can also be used.

© Beckhoff

Solutions available on the market always have their own configuration and programming interfaces. With 'Twincat Vision', configuration, calibration and programming are carried out entirely in 'Visual Studio'. This means that the user does not need to learn any other tools or special programming languages. Integration into the HTML5-based HMI is also ensured.

This means that configuring a camera in 'Visual Studio' is as easy as configuring an I/O node. All cameras that use GigE Vision as the bus and GenICam as the configuration standard are supported. By supporting this standard, cameras with special properties can also be easily integrated into the system. All cameras connected to a network adapter of the IPC are found via a scan. By assigning each camera to the description file (GenICam), all of its parameters are immediately available. In this way, there is no need to assign manual IP addresses or create extensive lists of parameters. After just a few moments, the cameras are ready for use and the real recording can be displayed in 'Visual Studio'. All parameters can then be set manually during the engineering phase or during operation using PLC function blocks.

If the image processing system is also to cover measurement tasks, the cameras used must be calibrated and the pixel sizes converted into metric sizes. Typical checkerboard patterns, circles or even three-dimensional calibration patterns can be used for calibration. The advantage here is that only a single image is sufficient for calibration: in the event of servicing, for example when replacing a camera and/or lens, this makes recommissioning easier and saves costs.

PLC programmer with vision know-how

As soon as the camera has been set up and calibrated, the actual programming can begin. The familiar PLC programming languages, C/C++ or Matlab/Simulink are available as programming languages with the solution integrated into the standard control technology. This will mean that PLC programmers will increasingly be able to solve image processing tasks without the need for vision specialists or knowledge of special programming languages. As a result, image processing will also become a natural function for automation - similar to how motion, safety or measurement technology functions have been integrated into control software in recent years.

The real-time synchronized automation applications of motion, robotics and vision - in this case image processing combined with the XTS linear transport system - open up efficiency potential for mechanical engineering and process sequences.

© Beckhoff

Vision programming starts in the PLC by adding a library. This provides functions and function blocks for acquiring an image, for processing and filtering the image data and for recognizing objects. It is also possible to measure objects in the image in this way. The recognition and identification of codes is particularly necessary in the field of packaging technology, which is also covered by corresponding library functions.

To transfer an image to the IPC, the image capture must first be triggered via the camera. The trigger event is usually initiated digitally - for example using an Ethercat output terminal with distributed clocks functionality. This allows synchronization to take place with an accuracy in the microsecond range. Once the image has been stored in the IPC, further algorithms are used for filtering. In order to be able to assess whether the filters are really working correctly, all intermediate steps can be displayed as an image in 'Visual Studio' or in the HMI.

Once the raw image has been prepared, the actual search and analysis algorithms are started. There are currently more than 500 different algorithms available in various categories. Within the PLC, these can be swapped as required using online change parameters and/or code without stopping the PLC. Compared to classic C++-based solutions, this represents a major advantage during commissioning.