A powerful 3D snapshot camera with a pre-installed and pre-trained neural network combined with a deep learning-based localization algorithm form the basis for Palloc. The system is able to learn on the basis of images and examples, develop evaluation procedures and incorporate human experience into the sensor in order to reliably recognize previously unknown features and continuously improve the measurement accuracy of the localization of cubic bodies such as boxes and cartons. Deep learning enables functional progress in automation technology, which also applies to the Palloc camera system. It can localize different variants of stacked boxes on pallets and provide 3D position coordinates (contour and height) for precise robot operation. The flexibility of the neural network allows new carton variants to be added on demand via a user-friendly AI tool suite. The neural network sensor app is integrated directly into the camera, eliminating the need for an additional PC and making system integration much easier. The system can also be seamlessly integrated into the control systems of various industrial robot and cobot manufacturers via the standard Ethernet TCP/IP interface.

Stereo and color image capture with 3D snapshot camera

Palloc uses the Visionary-S 3D snapshot camera with integrated structured lighting for precise image capture of the often tightly stacked crates. It enables a combined stereo and color image capture of cubic bodies such as crates or boxes, for example on pallets, and thus facilitates the segmentation of parts and the calculation of height information. The camera delivers up to 30 full-screen color images and 3D image pairs per second in high resolution. These 3D values from Stereovision, which are automatically offset against the contrast values of the color data, ensure that the contours, edges and layer heights of boxes and packages are captured and measured very precisely and reproducibly. As a result, the exact gripping point is transmitted to the control system for fully automatic robot guidance. Due to its mechanical design, the camera can be mounted either at the end of the robot gripper or stationary above the detection area and detects the smallest features - regardless of the height of the topmost position of the cartons to be picked.

AI integrated into the camera

The Palloc system is supplied with a pre-installed neural network that has already been trained with large amounts of data from different boxes in terms of size, color, design and printing. The Visionary-S camera thus functions as a fully automated system including smart software that adapts flexibly to the depalletizing application conditions. Neural networks can not only automatically identify similar contours, but also process deviating object variants and their data by evaluating graded degrees of abstraction of image details. In addition, Palloc makes it easy to add new box types via an intuitive AI tool suite such as Sick's dStudio web service, which is used to train neural networks. This allows easy expansion of functions without the need for expertise in image processing and robot programming.

Precise and collision-free robot guidance

The author: Maik Ahlers is Market Product Manager at Sick-Vertriebs-GmbH in Düsseldorf. © Sick

The AI-supported camera system was developed with the aim of automating manual depalletizing using robots. At the same time, the aim was to create a future-proof solution for various logistics and production applications that is easy to install, user-friendly, flexibly expandable and can be integrated into digitalized processes. Commissioning the system takes less than an hour and can also be carried out by people without in-depth knowledge. The system has generic interfaces to the robot controller or automation system (PLC) of various manufacturers as well as a web server user interface that enables seamless connection to the Industrial Internet of Things (IIoT) and IT systems. The new deep learning-based algorithm enables cost-efficient, robot-assisted depalletizing, in which cartons of different sizes and heights are localized in a matter of seconds. The system reports the contour data and height of the cartons for the flexibly adjustable gripping point to the robot controller so that the robot picks up the cartons individually or in groups as required and places them at a destination such as a conveyor belt or pallet.