Framence, provider of photorealistic digital twins, and robot manufacturer Anybotics are working together as part of the cooperation project 'T-IRIS' (Twin-Integrated Robotic Inspection Solution) to make the use of the four-legged industrial robot 'Anymal' as easy as possible for the untrained end user. At the same time, however, all the functions and capabilities of the autonomous robot should be available to users. For this purpose, a photorealistic digital twin is used as a user interface and data platform so that the planning of inspection work can take place directly within the photorealistic environment. In addition, the use of a photorealistic digital twin enables intuitive access to the inspection results transmitted by the robot.

Inspections by Anymal

Anybotics' four-legged robot can autonomously perform inspections in multi-story and complex facilities thanks to its AI-powered mobility. With its four legs, Anymal moves easily over uneven terrain and even climbs stairs with a gradient of up to 45°. With its robust design, impact protection and IP67 dust and water resistance, the robot is particularly suitable for industrial environments.

During its inspection missions, Anymal uses high-resolution sensors and machine learning to provide accurate real-time data, some of which has already been analyzed, which can be transferred directly to third-party systems via the Anybotics API. The inspection equipment includes a 4K zoom camera, a thermal imaging camera, an ultrasonic microphone and a spotlight as standard. The 55 kg robot has a built-in Wi-Fi module for data transmission; an LTE module can also be added. Anymal can carry out inspections in the factory for 90 to 120 minutes per battery charge and then return autonomously to its charging station.

Photorealistic digital twins as user interface

Framence's twin software is used to control the robot in the most user-friendly way possible. The company has developed a process for creating digital twins that depict reality photorealistically à la Street View. From simple photos taken with a digital camera, a true-to-scale, photorealistic digital twin of the overall environment of a location is created. Remodeling is not necessary. As the software is manufacturer-agnostic, information from third-party systems such as dashboards, instructions and videos, as well as inspection data from robots, can be integrated into the photorealistic twin and displayed on the respective system. Such data is often displayed in the system via so-called points of information (PoI), which are placed as a point with a unique 3D position on the corresponding system in the twin.

The integration

The inspection result of the kiln door can be called up immediately in the photorealistic digital twin. The user can call up, edit and save any data available in the image model via PoI.

© Framence

Users can send inspection commands directly to the four-legged robot via the photorealistic digital twin - these can be inspections of individual systems, for example, which can be carried out at any time and on an ad-hoc basis with just one click. This allows critical situations to be checked without having to put human lives at risk. Complex missions can be planned in advance from a large number of individual inspections in the photorealistic model, saved and executed both ad-hoc and cyclically. After successfully completing an inspection, Anymal automatically returns to the charging station where it waits for the next mission.

If a stable internet connection is available on the inspection routes, users can track the progress and results of the inspection in real time. However, as complex industrial plants in particular do not always have comprehensive internet coverage, the robot saves the inspection data in areas without an internet connection. As soon as an internet connection can be re-established (for example in the charging station area), Anymal transmits the data to the photorealistic digital twin for evaluation by the user.

The results of the individual inspections are automatically attached directly to the corresponding asset in the photorealistic twin using the PoI. There is also a mission tool that summarizes all processes and results. The entire system can be linked with additional logic and intelligence, which means that the inspection results are not only saved, but also analyzed and interpreted. For example, warning messages can be configured if recorded values are outside the norm.

Preparations for the pilot project

After climbing the metal stairs, Anymal inspects the furnace door with the thermal imaging camera.

© Framence

As part of the pilot project, Anymal was tested in a waste-to-energy plant. There are numerous areas in the boiler house there that can be physically demanding or even dangerous for employees due to noise and high temperatures. The use of a robot can be an effective addition here to ensure the safety of employees.

At the beginning of the pilot project, photos were taken of the areas in the boiler house where the inspection robot is to operate in order to create the photorealistic digital twin. Once these photos had been processed into the digital twin, the robot's mission was planned on screen. The systems to be inspected and the type of inspection - for example photo, video and audio recordings as well as temperature measurements - were defined. Once all relevant inspections had been defined in consultation with the operators of the waste-to-energy plant and summarized in various complex missions, commissioning could begin on site.

On the road at the waste-to-energy plant

Thanks to the preparations in the photorealistic model, Anymal was able to carry out complete robot missions on the twin after just a few hours.

One of the biggest challenges during commissioning were the sometimes very narrow corridors and the almost 45° steep stairs. These were measured and checked in advance in the digital twin and therefore ultimately posed no problem for the robot. This allowed numerous inspections to be carried out on different levels of the boiler house. In addition, a large number of analog and digital sensors, control wheels, valves and audio signals were recorded with the cameras and microphones of the four-legged helper and the corresponding inspection results were transmitted to the digital twin.

In view of the quick and easy commissioning and use of the robot on site and the intuitive operation of the digital twin as a communication and data platform, the pilot project was a great success. The objective of overcoming the 'adoption barrier' of robots in inspection and maintenance tasks with the combined solution of robot and digital twin was achieved.