The AI analyzes and evaluates potential gripping points. The robot then executes the selected grip.

© Siemens

Due to sensor noise and occlusions, gripping the robot over many objects is very challenging. Physical properties such as object shape, position, material properties, mass and the positions of the contact points between gripper and object are difficult to derive precisely. Recent results indicate that deep neural networks can indeed be used for such problems. Their training is based on large data sets of human grasping labels or attempts at grasping on a physical system. Successful grasps can be planned over a large number of objects directly from images without explicit modeling of the physics.

The creation of training data sets is very time-consuming. To shorten the training time, grasps can be calculated quickly over a large data set of object matrix models - physical grasp models help here. These methods rank the grasps according to grasp robustness - i.e. the probability of grasping success predicted by models from mechanics. The user can see whether the gripping processes can withstand any forces and torques or not. The probability distribution of properties such as object position and surface friction plays an important role here. In practice, these perception systems are slow and prone to errors. Therefore, the findings cannot be generalized well for new objects.

The evaluation of the gripping points: The potential pairs of gripping points pass through the neural network and are analyzed and then evaluated by the artificial intelligence.

© Siemens

In other words, the task of determining gripping points using conventional methods is not accurate and is only possible with a probability calculation. In industrial applications, however, estimated values are not sufficient to integrate them into a smooth process. This is where artificial intelligence comes into play, as it has the ability to solve the gripping of objects in a much smarter way. In the future, handling systems will be able to recognize and grip objects autonomously. Instead of trying to estimate the shape and position of 3D objects, the artificial intelligence 'Dex-Net 2.0' used by Siemens for this purpose uses completely different methods to enable optimal gripping.

Dex-Net 2.0 is an AI that is based on neural networks and was specially developed by the University of Berkeley (USA) to enable the gripping of unknown objects. Specifically, Dex-Net 2.0 uses a probability-based model to generate synthetic point clouds. It is therefore no longer necessary to estimate the shape and position of 3D objects from images. The software captures the robustness characteristics from data sets of 3D object meshes using physically based models of grasping, image rendering and camera noise. The key finding behind the method is that robust parallel jaw grasps of an object correlate strongly with the shape of the object.

The AI is executed on the new module for the Simatic S7-1500.

© Siemens

In detail, the gripping of arbitrarily shaped and positioned objects works as follows: The integrated 3D camera captures the shape and position of an arbitrary body. The digital image of the object is generated from this. This allows potential gripping points to be determined by analyzing the object. After running through the neural network, the AI analyzes and evaluates the optimal pair of gripping points using a rating system. The decision is therefore made for the pair of gripping points with the highest rating. And this forms the basis for the AI's decision on how the robot should grip the object - executed on the module for Simatic S7-1500. In the conventional sense, nothing more needs to be programmed, or to be more precise: nobody could program this complexity.

Because edge computing processes data where it is generated, AI can be bundled at the edge of local networks for entire systems.

Bundling AI via edge computing

This is what happened, for example, in PCB production at Siemens in Amberg. Around 6 million Simatic products are produced here every year. 75% of the value chain is automated, making the factory one of Siemens' leading production sites for digitalization. Previously, all printed circuit boards were sent through an X-ray machine to ensure that all products left the factory without defects. However, each X-ray process slowed down production due to the long throughput time, thus reducing efficiency and the number of units produced.

But what if the quality of each individual piece could be checked after various production steps, and the data collected, evaluated and fed back? Then it would be possible to tell which PCB should be X-rayed again before the X-ray machine and which is unnecessary. Thought, done: since 2017, Siemens in Amberg has been using 'closed loop analytics' on a line where the solder joints of bus connector PINs are X-rayed within the Simatic ET200SP base unit - in other words, analyzing data in order to use it in the next step and feed data back into the process in a controlled manner. This saves time and therefore money. To this end, special algorithms were developed to predict the probability of manufacturing errors.

In the first step, the process data was collected from the physical equipment and the X-ray results of the produced parts (quality labels). Subsequently, multi-variant dependencies of process data and X-ray labels were modeled using supervised machine learning algorithms (binary classification). The predictive algorithm is integrated into the IT landscape to enable continuous training of the machine model and thus increase the accuracy of the predictions. The use of these AI algorithms greatly minimizes the number of X-ray examinations. As they can be integrated directly into the production process on site, they can be continuously

be trained on an ongoing basis. The ultimate goal of 'Closed Loop Analytics' at the Siemens plant in Amberg is 30% fewer tests in the X-ray machine. In summary, it can be said: The S7-1500 TM NPU module integrates seamlessly into the automation system, making it easy to use AI algorithms directly in the controller. In the future, this will make it possible to solve tasks that cannot be realized with the classic automation methods used today, or only with great effort.

However, AI will not only find its way into the controller, but also into all levels of the 'Totally Integrated Automation' portfolio, thus enabling scalable solutions from the field level to the controller and edge level through to the cloud. The ultimate goal is to be able to provide an AI workbench that enables every automation expert to solve problems with artificial intelligence.

The example of the learning gripper robot also shows that it is no longer just about operational automation, but rather about how AI capabilities - such as ego awareness, intelligence and predictive competence - can advance industrial automation. In other words, automated production is becoming autonomous production based on cyber-physical systems that continuously optimize and organize themselves independently - in other words, make their own decisions.

Author:
Nelli Klein is Marketing Manager Future of Automation at Siemens.