In bin picking, a robot picks specific parts from a bin in order to make them available for processing or for further process steps. This involves placing the items, which are usually lying in the bin in a disorderly manner, in the correct orientation at the new storage location. Bin picking is primarily used in the production workflow, for example in the assembly of individual parts, separation or packaging of finished products.

Another area of application is intralogistics. Here, for example, a wide variety of objects are stored in boxes (bins) in fully automated high-bay warehouses. To pick the parts, autonomous industrial trucks drive to the shelves, load certain bins and transport them to the picking point. There, permanently installed robots remove the relevant items from the bin and prepare them for the rest of the process chain. The robots are connected to 3D cameras and image processing software in order to recognize and pick the objects accurately.

A wide variety of object geometries

The objects to be gripped can have a variety of different geometries and shapes. To ensure good recognition rates, the machine vision system would need a detailed CAD model of each individual object. In many application scenarios, however, CAD models are not available.

CAD model-free approaches

MVTec offers the new "3D Gripping Point Detection" feature for these bin-picking applications, which is based on a pre-trained deep learning network. The particular benefit of the technology lies in the extended application possibilities of bin-picking processes: it can be used to grip objects whose shape is not known in advance. The process is able to robustly identify possible gripping surfaces on absorbent objects so that the vacuum cup can safely pick up the object. No CAD model or other knowledge about the shape of the object is required for the gripping process. This means that bin-picking applications can be extended to numerous object categories with different geometries. In addition, the robot can also robustly grip flexible, deformable objects that are not described by a rigid shape.

"3D Gripping Point Detection" is integrated into the current version 22.11 of the standard Halcon machine vision software and is therefore part of the Halcon Toolbox. This includes image acquisition, the HDevelop development environment, various language interfaces and, at its heart, an image processing library. The software also offers the option of running applications on a wide range of embedded platforms. The image processing library contains a bouquet of different methods - from camera calibration, code reading, 2D and 3D measurement, filters and blob analysis to various deep learning methods such as classification and global context anomaly detection. The individual tools from the library can be combined flexibly and efficiently in order to respond to the specific requirements of an image processing application.

The complete machine vision workflow

Halcon covers the complete workflow of any machine vision application. This includes image acquisition, pre-processing, processing, post-processing and results output. This allows the entire bin-picking process to be raised to a high level. For example, the 2D and 3D image data can be prepared using appropriate tools during pre-processing so that the processes in the main part are more robust. The classic steps in pre-processing include noise reduction and suitable background treatment, for example to reduce the image domain to the objects in the bin. In addition, suitable pre-processing is able to increase the speed and robustness of the process. For example, the image domain can be restricted or cropped accordingly. The software also supports dedicated AI hardware accelerators via the AI Accelerator Interface - via TensorRT or OpenVINO - which makes the inference of 3D Gripping Point Detection many times faster.

The author: Luzia Beisiegel is Product Owner Holcon Library at MVTec Software.

© MVTec

There are also plans to expand the technology in one of the upcoming versions to include post-training in order to be able to optimize the bin-picking process with regard to certain hardware conditions or environmental situations.