Gesture control is not a new development; it is already being used in some areas. For industrial applications, a few things still needed to be clarified. The Fraunhofer Institute for Machine Tools and Forming Technology IWU has now announced: Industrial gesture control is now ready for the market. The researchers are certain that gesture control of robots by humans will make their collaboration in production more intuitive, efficient and ergonomic. What's more, applicable guidelines on machine safety and data protection are complied with.

Controlling colleagues' robots with gestures

In the future, people will be able to control their colleagues' robots contactlessly with their arms and hands using gestures. "During development, this always reminded us of conducting a choir or orchestra. In our gesture control system, the members of the 'ensemble', namely the robots, also pay very close attention to arm and hand movements," explains Paul Eichler, project manager in the Department of Robotics at Fraunhofer IWU. "Conventional operating elements such as buttons and switches become superfluous. Employees can move naturally to interact with the robots - as if they were signaling a 'stop' or a direction to another person in the factory."

This is how the robot sees a human: A Fraunhofer IWU researcher demonstrates industrial gesture control from the perspective of a robot.

© Fraunhofer IWU

For touch-free robot control, the researchers use specially developed path planning algorithms, smart optical sensors, fast and stable image processing methods and freely placeable user interfaces - all coordinated with each other. "Our technology brings gesture control to the industrial sector. Many people are familiar with it from home, where it has so far mainly been used in computer games, such as consoles for home use. The player's movements are detected and immediately translated into game maneuvers on the screen. However, we are not controlling game characters here, but machines and systems," says Dr. Mohamad Bdiwi, Head of the Robotics Department at Fraunhofer IWU, explaining the principle. "Programming skills are no longer necessary for direct control. The employees control the robots intuitively."

Proven market maturity in e-car manufacturing

The gesture control of industrial robots is technically mature and reliable. It complies with the applicable safety requirements for certification in accordance with the EU Machinery Directive. Anonymization procedures are also integrated. As data is not stored in a cloud but processed on site in the factory, data protection is also ensured in accordance with the GDPR.

The fact that gesture control works reliably and safely and can also be introduced into industrial production has been proven in the body shop of electric vehicles at Volkswagen Saxony. The company sees clear advantages in the production of the ultra-modern modular electric drive matrix (MEB). IWU project manager Paul Eichler: "The motivation for our partner was to improve the ergonomics for the employees in the testing and work station at the end of the welding line in the body shop. We have shown that our system of gesture-controlled heavy-duty robots has many advantages in production. Gesture control enables employees to adjust the position and orientation of the robot individually and in fine increments. This makes production more efficient and flexible."

Gesture control also fundamentally facilitates collaboration between humans and robots. Although they now often work side by side without a safety fence, direct interaction was previously not possible. In close proximity to humans, industrial robots switch off for safety reasons. Now the two can work together directly and safely.

Focus on 'Cyber-Physical-Human Systems' at the Chemnitz site

Intuitively controlling production robots with gestures is one example of the innovative strength of the Chemnitz site in the field of cyber-physical-human systems (CPHS). The Fraunhofer IWU works closely with the Chair of General Psychology and Human Factors at Chemnitz University of Technology across the boundaries of scientific disciplines. There, research is being conducted into the cognition-based support of psychological processes and their implementation in technical systems using artificial intelligence algorithms.

Dr. habil. Franziska Bocklisch, head of the professorship, explains the joint research objectives with Fraunhofer IWU: "The close interaction between humans and technology in Industry 4.0 shows how important it is to combine technology and human sciences. In the future, many more complex tasks will only be able to be mastered by humans and technology working together. This poses completely new challenges for human-centered technology development, for example in the targeted support of human information processing in the operation and maintenance of production systems. In Chemnitz, we have pooled the necessary transdisciplinary expertise to ensure that humans and robots can work together harmoniously in the production process."