In contrast to industrial robots, mobile service robots operate in dynamic environments and often among people who are not familiar with them. In the inpatient care environment, they can also encounter particularly vulnerable people. The S³ research project has therefore developed technologies that improve the perception of objects and the environment as well as the handling capabilities of service robots, with a particular focus on functional safety. The project received funding from the Federal Ministry of Education and Research.

The work at Fraunhofer IPA focused on the further development of appropriate software technologies for environment detection, the safe gripping of objects and their integration on the 'Care-O-bot 4' service robot. A combination of radar sensor and cameras was used on the robot for this purpose. The radar sensor enables the robot to reliably recognize people. The camera data is evaluated using machine learning methods to determine what a recognized person is currently doing. By determining the body posture based on the joint positions, the software can assign actions such as walking or gripping. This enables the service robot to adapt its behavior to its human counterpart in such a way that everything runs safely.

The robot can recognize and pick up beverage bottles and take them to a desired destination, such as the kitchen of a living area.

© Fraunhofer IPA / Florenz Graf

The IPA researchers also improved the robot's understanding of its surroundings. Previous algorithms had problems identifying and localizing transparent objects in space with standard 3D cameras. The newly developed algorithms are now able to do this. Neural networks help the robot to better detect water bottles, for example, and determine their exact position. The basis for this was a test scenario in which the robot exchanges empty water bottles for full ones in a nursing home. On instruction, the robot moves to a desired resident's room, recognizes the bottle it is looking for, grabs it and takes it to the kitchen in the living area. Thanks to an advanced environment model, the robot 'knows' where the tables are so that it can specifically search for bottles in the residents' rooms.

Safe gripping was also developed further in the project. IPA experts have developed software that dynamically adapts arm movements to the recorded environmental data in order to avoid obstacles. For example, if a nursing home resident spontaneously reaches for a glass while the robot wants to pick up the water bottle next to it, it quickly reschedules its movement.

Practical tests in the nursing home

The IPA researchers tested the test scenario described above with all the technologies mentioned in a Bruderhaus Diakonie nursing home. For this purpose, Care-O-bot was given 4 new arms specially designed for collaborative use. They are specially padded and can stop automatically when touched. A rotating and tilting head unit ensures a safe all-round view.

The robot was in use for two weeks during the tests, repeatedly fetching drinks bottles from the residents' rooms and taking them to the ward kitchen. There, the staff refilled them and the robot returned the full bottles.
The robot found its way around the sometimes winding corridors of the care facility. If a person came towards it, it waited at a safe distance if necessary. It usually recognized the beverage bottles reliably and then picked them up without colliding with them. One difficulty was observed when refilling the bottles with chilled water. Condensation formed so that the gripper could no longer hold the bottle. In some cases, test runs also had to be aborted because otherwise staff would have been hindered, for example if the robot blocked narrow aisles.

Survey of nursing staff

In addition to the technical evaluation, the IPA researchers also conducted a survey of the nursing staff to determine the practical added value and areas for improvement of the robot's use. The nursing staff perceived the robot as safe and were not afraid of it. However, they would like the robot to complete its tasks more quickly. For long-term use, the robot should have more comprehensive interaction capabilities, for example by using facial expressions or speech to provide feedback on its current status and planned activities. The nursing staff would also like the robot to be smaller and more maneuverable.