Computer-based vision for robotic systems

Image processing

Zeljko Loncaric | Inka Krischke, 19.06.2019, 00:30

Computer-based vision for robotic systems

Computer-based vision is becoming increasingly important for robotic systems. But how can hard real-time applications and time-sensitive networking be combined with vision systems and artificial intelligence (AI)?

Images

Computer vision is a rapidly growing market with the potential to revolutionize a wide range of applications such as advanced driver assistance systems, medical imaging, precision agriculture, retail, advertising, media, security and surveillance, unmanned aerial vehicles (UAV) and robotics. The increasing demand for computer vision systems for new fields of application in combination with artificial intelligence is one of the main driving factors. AI is currently expected to achieve a CAGR of a whopping +47% in the computer vision market, while the overall machine vision market is expected to grow by 'only' around 8.5% by 2025. According to Researchstore.biz, the main growth drivers are the increasing requirements for quality inspection and automation, the further development of vision-controlled robotic systems, the increasing acceptance of 3D systems for machine vision and the rising demand for application-specific image processing systems that offer individual evaluation and processing functions in addition to automatic image capture.

New challenges

Providers of robotics and control solutions who want to implement their new applications with machine vision and AI algorithms face major challenges. They have to adapt the latest technology trends in a highly dynamic environment and integrate them profitably into their applications. One problem here is often generating the large amount of continuous data required to adequately train the AI algorithm. At the same time, however, all exceptions that may arise from interaction with humans, for example, must also be taken into account. There are also tasks to be solved, such as the Industry 4.0 connection of the systems - which must be comprehensively protected for this, which in turn requires increased attention, as networking always poses a risk of attacks. In the area of predictive maintenance, public LPWAN technologies such as Sigfox and NB-IoT are also creating new strategies for collecting relevant reporting values, resulting in a number of urgent tasks with which OEMs can and must differentiate themselves from the competition in order to keep pace with general developments.

Technology partner wanted

The AI-based vision platform for real-time robotics from Intel, Congatec and Real-Time Systems brings together heterogeneous partial solutions on a homogeneous solution platform and thus contributes to workload consolidation.

In this highly dynamic environment, OEMs are therefore looking for reliable technology partners for their embedded vision computing ecosystem as a foundation for their new solutions, ideally also German ones in an increasingly uncertain international environment. Providers such as Congatec and Real-Time Systems have developed a solution platform for vision, AI and real-time controls in collaboration with Intel. With its fast, deterministic behavior on up to six cores, the industrial application server platform based on COM Express Type 6 modules with an Intel Xeon E2 processor can handle multiple real-time and non-real-time tasks. The application-ready, multitasking-capable industrial control platform uses the real-time hypervisor from Real-Time Systems, which allows several operating systems to be run in parallel and completely independently of each other on one processor without affecting its response time. It is designed for the next generations of vision-based collaborative robots and automation controllers that need to handle multiple tasks in parallel - including situational awareness using deep learning-based AI algorithms.

Workload consolidation included

Smart embedded vision platforms with AI-based situational awareness are made up of many small functional modules whose interaction must be validated.

The main aim of this platform is to merge individual partial solutions into a homogeneous solution platform as well as to consolidate workloads. Previously separate systems for vision, AI or IoT gateways, for example, can be easily consolidated together with hard real-time applications and time-sensitive networking (TSN) on one hardware. This saves customers from having to develop and maintain different embedded systems for individual tasks, which offers enormous cost benefits. In order to develop such heterogeneous systems, OEMs primarily require a powerful real-time hypervisor as well as a multi-core embedded server platform suitable for the application, on which there are as many cores as are required for the specific tasks to be kept discrete on the software side.

The industrial-grade solution platform presented by Intel, Congatec and Real-Time Systems at Embedded World 2019, for example, is based on COM Express Type 6 modules with Intel Xeon E2 processors and integrates three application-ready, pre-configured virtual machines. One operates a vision camera from Basler, in which vision-based object recognition is carried out under Linux using the 'OpenVino' software from Intel. The AI algorithms are executed on an Intel 'Arria-10' FPGA card from Refexces.

The Intel distribution of the 'OpenVino' toolkit is based on Convolutional Neural Networks and supports the heterogeneous execution of deep learning inference algorithms across CPUs, GPUs and FPGAs as well as the Intel Movidius Neural Compute Stick.

The independent real-time partitions each run real-time Linux to keep an inverse pendulum in balance in real time, symbolizing the distributed manufacturing robots. Visitors to Embedded World, for example, could try to disturb the balance of these pendulums - the system reacted immediately in real time and kept the pendulums in balance.

Another Linux partition was used to run a secure gateway 'onboard'. The advantages of this secure gateway integration are, on the one hand, the cost and space savings for the external gateway. On the other hand, they protect the user from dangerous backdoors of an external gateway, so that the controller can be designed directly as an edge device with a vision app.

In order to demonstrate the independence of the applications and their real-time behaviour on a single server platform with several virtual machines, the Linux partition on which the vision system was operated in the demo could be rebooted, which had no effect on the virtualized real-time system. In principle, all other operating systems can of course also be rebooted independently of each other on their respective virtual machines and checked for correct operation using a watchdog.

The partners are crucial

OEMs benefit from such application-ready solution platforms from a significantly reduced development effort, as many functionalities have already been tested and the interaction of the individual components validated. The fact that Real-Time Systems is also a Congatec company also facilitates the scalability of the solution with other performance levels, because the fact that the 'RTS hypervisor' supports all common x86 platforms does not mean that every platform from an embedded computing supplier has been validated and tested in use together with the hypervisor. The same applies to components such as the vision cameras. For this reason, Congatec has entered into a cooperation with Basler with the aim of offering customers coordinated components for embedded vision applications.

Designed to meet requirements

If required, Congatec offers these individual, coordinated components as a fully developed, production-ready solution platform, including all certifications required for production-ready delivery to the end customer. Customers thus benefit from simplified handling and accelerated design-in of the embedded vision computer component as well as optimized service and support conditions.

Author:
Zeljko Loncaric is Marketing Engineer at Congatec in Deggendorf.

You might also be interested in

Edge AI + Vision Alliance

Quo vadis image processing?

Innovations in the fields of artificial intelligence and embedded vision as well as special processors and development environments for image processing open up a broader spectrum for the technology. An overview.

Image processing

Autonomous thanks to artificial intelligence

Machine vision for reliable quality assurance via plug & play and without a great deal of specialist knowledge - conventional image processing systems are generally a long way from this ideal. A German-Israeli start-up wants to change this.

Machine Learning

A direct comparison of solutions

The topic of machine learning raises a number of questions: Which data should be analyzed using which methods? What role does the user play in the data analysis process? And what about the real-time capability, explainability and reliability of the...

Sensors

The benefits of artificial intelligence for sensor technology

With the technologies and processes of deep learning, industrial sensor technology is on the verge of a far-reaching leap in functionality. Image-based sensors in particular are a field with great potential.

Lightweight robotics

Which gripper for which application?

Advances in the field of end-of-arm tooling (EoAT) make it possible: grippers and sensors are becoming increasingly intelligent and easier to operate. This expands the range of applications for collaborative robots and quickly pays for itself.

Machine Learning

Artificial intelligence requires computing power

The integration of AI in industrial applications is increasing the need for computing solutions that can cope with the complexity of the tasks involved. In addition to the CPU, processors specialized in AI algorithms and large amounts of data are...

Cloud solution

More usability through AI?

Flexible and cost-transparent booking of temporarily required resources - a promise that cloud providers often fail to keep: Although the offerings are comprehensive and powerful, they are just as confusing and lack transparency in terms of price. A...

Artificial intelligence

The arrival of AI in mechanical engineering

The number of real AI projects in mechanical and plant engineering is still very limited. This will soon change with a new generation of engineers and, above all, with the market launch of corresponding development tools.