Different types of vision systems, from classic to embedded.

© Embedded Vision Study Group

Embedded vision systems are transforming into decentralized, intelligent and autonomous participants in the production and automation environment. In the future, they will be able to automatically log into the network and know which network participant requires their measurement results. The automation network can then retrieve specific information from the device, such as image and signal data.

Thanks to their high computing power and intelligent algorithms, embedded vision systems will make it possible to evaluate data such as product, quality and process information in real time on site, process it for further use and report the results back. The systems therefore not only provide information, but also control processes in order to improve the performance, efficiency and quality of a production process. Preventive maintenance, human-robot collaboration and flexible system control for single-item production can thus be implemented more efficiently.

Parallel data processing in real time

What could an embedded vision system look like that is able to reduce the high load of increasing bandwidths in order to process and evaluate images decentrally in real time and at the same time reduce the load on computers?

Types of embedded vision systems: Embedded camera head consisting of sensor and FPGA (top), embedded vision system as smart sensor with FPGA SoC (middle) and vision SoC, mostly integrated in individualized applications (bottom).

© Embedded Vision Study Group

FPGAs (Field Programmable Gate Arrays), which can be found on frame grabbers as well as in embedded vision systems such as cameras or vision sensors, promise the greatest leaps in increasing process performance. A large number of integrated circuits can be implemented in an FPGA, which take over image processing processes such as image optimization, image evaluation or the generation of control signals. With their ability to process data with very high parallelism, FPGAs offer optimal conditions for the evaluation of image data in real time. This allows not only barcode reading to be implemented in a camera, but also the recognition of barcodes on products and image enhancement by eliminating unwanted reflections, soiling or geometric distortions.

FPGAs control sensors, pixel formatting and processing as well as the transfer to interfaces. They enable pre-processing in order to reduce the bandwidth of the image data or to carry out evaluations in order to obtain meta or control data. This allows many companies to incorporate their application know-how into the devices and address new applications and markets.

Whereas FPGA hardware programming used to be considered complex and expensive, it is now much easier to carry out using a graphical user interface via operators and data flow diagrams and without any hardware programming knowledge.

Since FPGAs are reprogrammable, any number of special applications can also be used in embedded vision systems. FPGAs can be coupled with processors in the form of a hybrid architecture. Such cyber-physical components represent a small computer, often consisting of embedded 'systems on a chip' (SoC) including special processors, special real-time capable microcontrollers and high-tech memories with high performance and minimal power consumption as well as multi-core architectures.

Heterogeneity calls for standards

The Embedded Vision Study Group (EVSG) aims to make the heterogeneous system architecture for embedded vision more permeable and has identified three technology fields (SC1, 2 and 3) for which interface standards are to be developed.

© VDMA

The Embedded Vision Study Group (EVSG) has taken up the cause of making this heterogeneous system architecture more permeable. It presented its report at the G3 Future Standards Forum (FSF) in Chicago in summer 2015. By further developing standards such as OPC UA and Gen<I>Cam for embedded vision in several working groups, the EVSG is driving standardization forward so that recorded (image) data can be processed without loss and evaluation results can be transported further.

In the EVSG report, three technology fields were identified as 'Standard Candidates' (SC) for which the working groups will develop interface standards:

• SC1 deals with the modular structure with sensor boards and processor unit/system on a chip (SoC) and their compatibility.
• SC2 concerns the software model (API) for communication with the embedded components and their control.
• SC3 deals with their integration into an automation or processing environment.

Three fields of technology

One specific question in the SC1 technology field is, for example, how a sensor can be connected to processor intelligence so that it automatically feeds its image data into a network. Until now, proprietary software programs have been used to control cameras and sensors or to transfer processing results as well as for device recognition and control. Interface standards must therefore be developed that are compatible with Gen<I>Cam in order to efficiently embed image processing devices in the production environment. The EVSG has evaluated various interfaces between sensor and processor/SoC, in particular MIPI, PCI Express and USB3, but has not yet been able to make a concrete recommendation for one of the technologies for reasons of long-term availability and technical limitations.

An important basis for the SC2 technology field is that embedded vision systems pre-process images internally, in some cases up to the level of result data. This new type of data requires extended generic description models of data formats and structures as well as their semantic information. The EVSG recommends a Gen<I>Cam standard extended to include the requirements of embedded systems, with a focus on supporting generic data formats and the processor modules that process the data. Security mechanisms such as data encryption and IP protection are an additional aspect to be considered here.

In the SC3 technology field, the transfer of pre-processed and compressed image data into concrete information remains a challenge. This requires standardized interfaces between embedded image processing systems and automation technology as well as the control and planning level. RAMI 4.0, the Industry 4.0 reference architecture model, recommends the standardization protocol OPC UA as the sole solution within the framework of a companion standard that offers a tried and tested software model. As a technology candidate, Gen<I>Cam was evaluated as a companion standard with a focus on semantics (adoption of the Gen<I>Cam SFNC - Standard Features Naming Convention), which the EVSG also supports. The image processing systems should then be integrated directly into the PLC software and thus into the production line. As part of a planned OPC UA specification, there will be a real-time extension in the future.