Industrial manufacturing is undergoing a profound upheaval: the shortage of skilled workers, intensifying competition in an increasingly globalized economy and rising demands in terms of product quality are presenting companies with new challenges. A willingness to innovate is one of the basic prerequisites for staying on course for success in the face of this change. More and more companies are therefore turning to new technologies and automating parts of their production lines. Robots are increasingly being used, which can now also perform demanding tasks that were previously the sole preserve of humans. Thanks to artificial intelligence (AI), they can now be equipped with a 'sense of sight' that enables them to capture and process images. It allows objects and structures to be recognized and parts to be handled as required. For a long time, such systems were too expensive and too complex to implement in small and medium-sized companies. In the meantime, however, they have become a cost-effective and easy-to-implement alternative to manual work that is increasingly being used in medium-sized companies.

Two-dimensional or spatial vision

Visitors to the Deutsches Museum in Munich can experience the interaction between robot arm, gripper and vision system up close. The application is based on a UR5 robot from Universal Robots, which is equipped with a gripper from Schmalz and a vision system from Robominds.

© Universal Robots

Automated applications with a vision system are based on a robot arm that is supplemented by a suitable peripheral device. It is linked to an image processing system that captures images using one or more cameras, which in turn are digitized by corresponding software. The system searches for previously defined features, such as the surface structure of an object, and triggers a specific action in the robot. For example, a component to be handled is fed to the next processing step or sorted out if it is faulty. Adequate lighting must be ensured so that the camera can correctly capture the key features of the parts.

Depending on the task at hand, images can be captured and processed in both 2D and 3D. Images in 2D are particularly suitable for applications in which objects, geometries or patterns need to be analyzed. In this case, the vision system uses the contours of an object to calculate a two-dimensional data set so that the cobot can identify the position, size or orientation of a part. More complex tasks, on the other hand, require spatial vision - an ability that comes into play, for example, in the area of assembly or when feeding machines with unsorted parts. Appropriate sensors enable the robot to perceive its surroundings in three dimensions.

Vision system based on the human model

The employees at Jenny | Waltle program the system independently for new orders and continuously optimize grippers, clamps and trays.

© Universal Robots

The Munich-based start-up Robominds shows how vision systems can successfully enable robots to interact with their environment. Founded in 2016, the company has developed 'Robobrain.vision', an AI system that is compatible with robots from various manufacturers and a wide range of parallel and vacuum grippers. Image processing is carried out using a 3D stereo vision camera, a connected computing unit and a software solution that can be adapted to individual requirements. A color image in 2D is combined with a depth image in 3D. The system, which can be quickly integrated into production lines, is designed to independently calculate the gripping points that the robot arm should approach. The software can therefore also quickly detect unknown objects without lengthy teaching processes - even if they are arranged randomly above and next to each other or have different surfaces or geometries, as is the case with bin picking.
"When developing our AI, we took humans as our model," explains Tobias Rietzler, technical CEO and one of the founders of the start-up. "Our technology is based on the principle of abstraction: the image processing system learns how to deal with bodies and geometries like a human brain and can later transfer these skills to similar objects. This is how we make robots intelligent."

Reaching into the box

A look at Austria shows that collaborative robots in combination with a 3D image processing system are proving their worth in industrial practice. Jenny | Waltle, a specialist in aluminum, metal and plastic parts based in Vorarlberg, uses this technology to tackle the problem of a shortage of skilled workers. The medium-sized company with 50 employees has been using two collaborative robots from Universal Robots since 2018. The two 'UR5' cobots work in close proximity to their human colleagues in bin picking and feed a CNC milling machine with parts.
For this purpose, pre-sawn aluminum parts, which lie unsorted in a box, are first scanned by a camera. Software then generates a 3D data set, known as a point cloud. This enables the first cobot to recognize the complex surface structures and the exact arrangement of the objects. Equipped with a vacuum gripper, it then removes part by part from the container. An additional axis on the tool flange enables the cobot to pick up the workpiece precisely and without collision. For maximum precision in the grip, it then aligns the part in a buffer.
If the cobot has picked up an object upside down, for example, it throws it back into the box and tries again after the next scan. If the part is positioned correctly, the 'UR5' places it in another tray. This is where the second cobot takes over, placing the components precisely in the hydraulic clamp of the CNC milling machine. After processing by the machine, it picks up the parts and places them in a final tray, from which the first cobot then throws them into an empty box. Up to 2400 aluminum parts are processed in this way every day in two-shift operation. The cycle times are between 30 and 40 seconds.