Until now, the automation industry has mostly operated independently of the machine vision industry. This may be due to the fact that the two worlds 'tick' very differently and have separate histories: While the roots of modern automation go way back to the 1960s and 1970s, machine vision is a comparatively young technology that has developed considerably in the last twenty years. Many automation specialists have always worked with tried-and-tested control and sensor systems and were less open to modern, 'seeing' processes.

Easier handling

With 'Merlic' from MVTec, users can create sophisticated image processing applications even without specialist knowledge of machine vision.

© MVTec

But now, with B&R and Beckhoff, two renowned suppliers are integrating image processing technology seamlessly into their control concepts for the first time. Machine vision providers are watching this development with interest and are creating solutions on the basis of which the two disciplines can grow closer together. Users benefit from the resulting synergy effects, for example through simpler, faster and more precise handling processes. As a provider of standard machine vision software, MVTec cooperates closely with companies from the automation industry, for example, and various sensor manufacturers integrate the company's software solutions into their smart sensors as standard.

The special thing about the integration of machine vision and automation is that the boundaries between the two system worlds are constantly shifting - the prevailing trend is towards simplified handling. This means that even users without in-depth programming knowledge should be able to create and implement image processing applications quickly and easily. MVTec is pursuing this approach with its 'Merlic' software, for example, which contains intuitive, user-friendly tools for this task. Coding experience is not required. The respective tools - for measuring or counting objects, for example - are displayed in an image-based format and guide the user through the entire application.

The software can also be seamlessly integrated into different hardware setups and PLC or production environments thanks to interfaces to common elements of machine vision peripherals and support for standards such as OPC UA, GigE Vision or GenICam.

Embedded Vision

Modern machine vision applications run on common embedded systems such as the ARM architecture.

© MVTec

Embedded systems are a growth market that both automation providers and machine vision providers have to deal with. Users should be able to benefit from user-friendly, simple and robust embedded vision solutions and corresponding tools. To this end, developers must be supported in the creation of applications on embedded devices. Compatibility with embedded systems such as the widespread ARM architecture is particularly important. 'Halcon' from MVTec, for example, runs as standard on ARM-based platforms with the Linux operating system without any special porting. The vision software can also be ported to embedded platforms such as Android, BeagleBoard-xM or DragonBoard. In this way, companies can make full use of machine vision functions even on compact hardware devices.

Modern machine vision solutions also make it possible to develop image processing applications on a standard platform (such as a PC) and then run them on an embedded system. This is because a larger monitor and more convenient input devices such as a keyboard and mouse are available on a desktop PC. Nevertheless, the entire machine vision library, including all tools, can be used on the embedded device. This is ensured, for example, by a special software engine that also runs on the devices and also supports remote connections for debugging. This simplifies the entire embedded vision development process.

Deep learning on embedded devices

In the smart factory, machines, robots, transfer and handling systems, sensors, image capture devices and other components communicate seamlessly with each other.

© MVTec

The aforementioned usability aspects also apply in particular to the use of new technologies such as artificial intelligence (AI). AI applications are generally very complex and time-consuming to use; they also require in-depth specialist knowledge on the part of the user. It is therefore advantageous if machine vision solutions already offer comprehensive, AI-based deep learning functions 'out of the box'. This allows self-learning algorithms to be used for demanding classification tasks without a great deal of configuration effort and at the same time significantly reduces the training effort, for example by using an already pre-trained neural network.

In terms of seamless vision integration, corresponding applications can also be easily ported to embedded platforms. For example, MVTec has successfully tested the deep learning inference of 'Halcon 17.12' on the embedded board 'NVidia Jetson TX2', which is based on 64-bit ARM processors. This means that users can also take advantage of deep learning on the NVidia board. This is made possible by the availability of two ready-made networks, which are supplied as standard. One of these, the so-called 'Compact' network, is speed-optimized and particularly suitable for embedded boards. In addition to deep learning, the full functionality of the standard image processing library is also available on the embedded devices.

Interaction of machine vision and PLC

Vision integration in practice: 'Merlic' supports standards such as OPC UA, GigEVision and GenICam. This simplifies integration into PLC and production environments.

© MVTec

Last but not least, the automation and machine vision sectors must also grow closer together in order to meet the requirements of Industry 4.0 and the Industrial Internet of Things (IIoT). End-to-end networked and highly automated process chains are required here - and machines, robots, transfer and handling systems, sensors, image acquisition devices and other industrial components must communicate seamlessly with each other. In this context, the smooth interaction between image processing and PLC is of great importance - common norms and standards are required here.

This is where the Open Platform Communications Unified Architecture (OPC UA), one of the most important M2M communication protocols, comes into play. The special feature of the current OPC UA specification is that it contains a comprehensive information model for describing machine data that enables semantic interoperability between networked components for the first time. All machine data such as control variables, measured values or parameters are not only transported, but also described semantically so that they can also be read by machines and robots. MVTec's software solutions support the OPC UA protocol as standard. The company is also actively involved in the VDMA's OPC Vision Initiative and is participating in the development of the 'OPC UA Companion Specification for Machine Vision'.

Author:
Dr. Maximilian Lückenhaus is Director Marketing + Sales at MVTec Software in Munich.