Tekniker specializes in the development of technology solutions for automation, industrial robotics and sensor technology, among others. The company serves sectors such as aerospace, the automotive industry and mechanical and plant engineering. One of its customers is the CIKAUTXO Group, a Spanish manufacturer of vibration-damping and fluid-guiding components for the automotive industry. On behalf of the company, Tekniker has developed an image processing-controlled robot system that can be used to automatically palletize components as part of industrial production processes. The robots remove disordered components from a box and then place them in a container that is specially adapted to the geometric characteristics of the parts. They are then ready for further processing in the production workflow.

Fully automated part gripping by robots is a major technical challenge. One particular difficulty lies in the fact that the components are not always delivered to the robot in an orderly fashion, but unsorted and even lying on top of each other. This means that humans often have to intervene.

Relieving employees through process automation

"Previously, the palletizing of parts was carried out purely manually. Our aim was to fully automate this process in order to speed it up, increase productivity and save costs. We also wanted to relieve our workers of monotonous, ergonomically unsuitable tasks so that they could devote themselves to more demanding tasks," explains Kepa Laka, Head of Automation and Robotics at CIKAUTXO. One option for implementing process automation was to implement a mechatronic system. However, this approach offers a low level of flexibility and reliability and also generates a high level of noise pollution. For this reason, those responsible at Tekniker decided to use a solution based on industrial image processing.

The system setup consists of two industrial robots from Kuka, a 3D scanner from Photoneo, a 3D depth camera from Intel, a PC running the image processing software 'Halcon' from MVTec and a Simatic PLC from Siemens to control the cell. One of the robots first removes the punched parts in large quantities from the input container using a magnetic multifunctional gripper and places them on two conveyor belts. Removal is stopped as soon as a certain volume of parts has been reached on both belts. The volume is calculated using the depth data provided by the depth camera and the customized Halcon algorithm, which combines 2D and 3D vision tools. The image processing software on the PC also continuously processes the data from the 3D scanner and provides the robots with the location and position of the parts that can best be gripped on the conveyor belt.

The second robot uses this data to start picking the parts from the conveyor belts and aligning them with the help of 3D vision so that they can be sorted into two target containers. After the first robot stops picking from the input bin, it 'helps' the second: It also grabs parts from the conveyor belt, aligns them and arranges them in the target containers. This works at a speed of eight seconds per part. In the target container, the parts must be inserted into the guide rods with the front facing upwards and with a tolerance of 0.5 mm. If the gripped part is pointing downwards, it is first placed in a rotating device in order to pick it up again in the correct position.

Precisely locate disordered parts

The point cloud in the image processing software shows the various components found (in blue), the best component to be gripped (in red) and the second best component to be gripped (in green). The robot's gripper in white.

© Tekniker

The particular challenge in this setup is for the robots to precisely recognize the positions and orientations of components in 3D space in order to be able to pick them up and put them down again safely. "To meet this requirement, we developed the Smartpicking software and integrated it into the overall solution. The software is able to precisely identify objects by analyzing 3D data and using the surface model and the corresponding geometry information. At the heart of the Smartpickin solution is the MVTec Halcon image processing software. It provides sophisticated algorithms and tools to precisely localize the parts lying chaotically on the conveyor belt and transmit the coordinates of the removal point to the robots," explains Jorge Molina, developer for intelligent and autonomous systems at Tekniker.

Specifically, classic 3D-based image processing methods are used. This primarily includes the 'Surface-based Matching' feature, which is integrated into Halcon. The technology uses data from 3D point clouds recorded by the 3D scanner. The data is further processed using various filters and pre-processing operators to improve the positioning of the parts. This enables robust positioning of the objects - regardless of the nature of the surface. The robots can now recognize the parts and their exact position and grip them accurately. This works for a wide range of different components, as they occur in industrial production processes, so that reliable position determination is guaranteed even for deformed or smooth surfaces without distinctive edges, which do not show any significant gray value differences in conventional images. In addition, the robots must also be able to precisely determine the relative position of the 3D camera in relation to the respective object in their coordinate system for a safe gripping process. For this purpose, Halcon is used to carry out a so-called hand-eye calibration, for example with an associated calibration plate. This calibration is essential in order to establish the relationship between the camera, robot and the parts to be gripped. Only then does the robot 'know' exactly where its gripper is in relation to the object to be gripped - i.e. where it has to move it to avoid reaching into the void.

Avoid collisions

The editor included in the Tekniker software can be used to define the point at which the component is to be gripped by the robot.

© Tekniker

In addition to detecting and determining the position of the parts to be gripped, possible collisions between the robot arms and individual objects (such as components or containers) must be avoided throughout the entire pick-and-place workflow. The '3D Object Processing' technology integrated in Halcon supports this process by identifying those parts that are favorably positioned on the conveyor belt and can therefore initially be gripped without collision. Based on this initial selection, the software then uses various decision criteria - such as the height and orientation of the parts or the position of the gripping point - to determine the optimum candidate and returns the coordinates of its position to the robot. This enables the robot to carry out the next gripping process safely.

The controlled process allows the removal of the components to be controlled in such a way that collisions and thus disruptions to the workflow are prevented. The software also recognizes the remaining capacity of the target container. To do this, operators precisely calculate its volume. A predefined threshold value can be used to determine whether further parts can be fed into the container.

The article was created based on templates from MVTec.