Brain-machine interfaces measure brain signals and use them to derive control commands for computers, machines or robots. DFKI is one of the pioneers in the mobile multimodal use of EEG data for interaction with robotic systems. Experts from the fields of multimodal interaction, embedded brain reading and interactive machine learning are working together in the newly launched Expect research project.

As part of the project, the Robotics Innovation Center and the Cognitive Assistance Systems department are developing an adaptive, self-learning platform that enables various types of active interaction and is able to deduce human intentions from gestures, speech, eye movements and brain activity. The project kick-off took place as a virtual event on July 20, 2020.

DFKI CEO Prof. Dr. Antonio Krüger, head of the Cognitive Assistance Systems research department and project manager: "In Expect, DFKI is bundling competencies and expertise from the fields of cognitive assistance systems and robotics at the Saarbrücken and Bremen locations. We are thus creating a cornerstone for further application-oriented projects based on research work on the integration of EEG data in hybrid brain-computer interfaces."

"In the future, AI systems will not only support people in limited areas. Rather, they will act like personal assistants. In order to fulfill this task, it is imperative that they understand people, deduce their intentions and provide context-sensitive support. Brain data opens up the possibility of giving robots a better direct insight into human intentions. An important area of application is, for example, rehabilitation after a stroke," says Dr. Elsa Kirchner, project manager for the Robotics Innovation Center research area.

One of the biggest challenges in human-robot collaboration (HRC) is natural interaction. In the field of human-computer interaction (MCI), the word "natural" does not describe the interaction interface itself, but rather the user's process of learning the various interaction principles of the interface. This learning process should be as simple as possible for the user, based on familiar (natural) interaction principles such as voice commands, gestures and the like.

This type of interaction is of great importance in the field of human-robot collaboration. It makes collaboration as efficient as possible and helps to reduce the fear of contact and general mistrust of human collaboration partners towards robotic systems in everyday working life. Expect therefore has a direct positive influence on the increasing collaboration between humans and robots in the world of work.

To make this possible, the project is developing methods for automated labeling and joint evaluation of multimodal human data and evaluating them in test scenarios. The systematic experiments also serve to investigate the question of how fundamentally important brain data is for predicting intentions in humans.

Maurice Rekrut, project manager for the Cognitive Assistance Systems research area: "In the Expect project, we are investigating the extent to which we can find correlations between different modalities in brain activity and thus make human-robot interaction more intuitive and natural. On the one hand, we want to design purely brain-computer interface-based interaction concepts, but also multimodal concepts in order to be able to provide the best possible interaction option in every situation."

The technologies to be developed in Expect can be used in many different ways - for example in space travel and extraterrestrial exploration, robot-assisted rehabilitation and assistance as well as in Industry 4.0 and industrial HRC.

Expect is being funded by the Federal Ministry of Education and Research (BMBF) as part of the ICT research program under the funding code 01IW20003 with a volume of 1.6 million euros over a period of four years.