Robot with LiDAR laser explores danger zones

In the event of disasters such as an accident at a chemical plant or flooding, the emergency services need to gain a quick overview. However, they are often not allowed to enter the site to avoid putting themselves in danger. The Fraunhofer FKIE in Wachtberg is working on a solution for this in the 3D-InAus project. A robot with a LiDAR laser (Light Detection and Ranging) explores the terrain. LiDAR technology scans the surroundings with pulsed light to measure distances. This creates a 3D model with buildings, interiors, open spaces, objects and all associated dimensions and distances. Users move freely around the 360° visualization using a joystick and examine the virtual environment.

Timo Röhling, Technical Project Manager from the Cognitive Mobile Systems department, says: "Compared to robotic systems that explore a danger zone with cameras, our project goes a big step further. The laser pulses provide measured values for the exact 3D mapping of a terrain or building. Distances and dimensions are not estimated, but determined to within a few centimetres."

Geometric point cloud from laser pulses

The core of the hardware is a LiDAR laser on a turntable. A rotating mirror is installed in the LiDAR module, which can scan a section of 16 vertical panes ten times per second. The turntable then rotates the laser so that the vertical sections horizontally cover a complete all-round view. A total of 1.3 million laser pulses are generated per second. These are reflected by the surroundings and the respective distance is calculated from the time difference. The LiDAR module is mounted on a vehicle that moves through the terrain in stop-and-go mode or continuously. Continuous driving is significantly faster, but less accurate. This creates a 3D point cloud, with each point representing a laser pulse or distance measurement.

There is also a camera system with up to six cameras. Their images are used to colorize the corresponding objects or shapes. "We merge camera images and point clouds, so to speak. This creates a vivid, detailed and at the same time geometrically correct 3D environment with houses, open spaces and objects," explains Röhling.

In the visual representation, the point clouds are colored by camera images. This creates a virtual 3D environment that combines vividness with exact geometric information. You can move freely in the virtual scenery. © Fraunhofer FKIE

The raw data obtained in the LiDAR laser is pre-processed in a computer module in the robot during the journey. The final visualization then takes place stationary in post-processing. A 400 m × 400 m area can be mapped in around three hours. If things need to be done particularly quickly in the event of a disaster, an initial overview can be obtained after just one hour thanks to accelerated operation. It is also possible to send several vehicles on a reconnaissance tour at the same time.

3D mapping is also valuable for the German Armed Forces, which commissioned the research project. It can be used to generate complex situational images of unknown terrain or a danger zone - and thus protect the lives of soldiers. The system's software is able to process the measured values from sensors that detect gaseous toxic substances or radioactive sources. These can be placed on the 3D maps.

Virtual GPS in the building

The robot system is usually controlled by radio, with users using a joystick and tablet. If there is no radio contact, the robot systems can also move around the site automatically.

Exploration in buildings poses a challenge, as there is no GPS reception here. The researchers at Fraunhofer FKIE have also found a solution to this problem: By mapping the terrain, the position and size of the building are known, from which the software generates a virtual GPS for the inside of the building. This enables the robot system to navigate independently inside the building.

Flexible platform for different scenarios

The research team made sure that the robot system is as versatile as possible. The laser module and turntable can therefore be mounted on a wide variety of vehicles. Depending on the terrain, land vehicles with wheels, tracked vehicles or even drones can be used.