3D sensing technology is considered a key to natural and intuitive human-machine interaction. The contactless operation of devices via eye tracking, gesture and facial recognition makes a new type of consumer goods, security features and industrial solutions possible. And thanks to mature technology, affordable prices and the corresponding market demand, 3D sensing technology is now ready for the leap into the mass market.

Perceptual computing combines the capture of information with its interpretation. The basic prerequisite for this is the sensor technology that supplies a machine with image data in 2D and 3D. This must be followed by a processor that evaluates this data and makes decisions. Depending on the application, this can be a GPU, an FPGA or an ASIC. Intelligent algorithms that recognize objects, gestures or faces, for example, can be implemented on these platforms. In an industrial environment, for example, production and transport processes can be monitored and subsequently optimized. It is possible to guide industrial robots that react to their environment - for example to a person moving into the production area - and avert the risk of injury.

Application examples

Perceptual computing enables intelligent applications across all areas - for example for drones and consumer devices as well as for industrial and home robots: coffee machines can recognize and identify users and their drinking preferences. People can operate devices with gestures, in a way that they normally interact with other people. Cleaning robots will be able to analyze their environment and thus clean the home in an intelligent way, avoiding obstacles and preventing valuable items - such as earrings - from being vacuumed up.

Perceptual computing supports virtual and mixed reality (VR/MR) in the fields of gaming and virtual collaboration by intuitively merging real and artificial objects. 3D sensing enables industrial robots to pick up and place objects like humans and prevent collisions during this movement. This makes human-machine interaction effective and almost self-explanatory. Drones can follow people and recognize obstacles. Surveillance and security applications can use a wide range of technology and data to navigate, control and interpret a variety of situations. Current 3D sensing products such as Intel's RealSense technology line further enhance perceptual imaging applications.

.....without the need for a separate graphics or host processor.

© Framos

The 'RealSense' technology is based on stereo vision. Two camera images are recorded in the infrared range and then searched for identical pixels on both images. The distance between the two sensors is used to triangulate and thus determine the depth information of the pixel in question. With this technology, OEMs and vision engineers can implement human-like 3D perception in their applications without the need for a completely new development. Imaging and vision experts, such as specialized distributors and system integrators, can help users leverage 3D sensing technology to implement advanced applications and make machines see and think.

RealSense technology comprises a series of components that - when integrated into devices or machines - enable them to 'understand' and interact with their environment in three dimensions. The product line includes a vision processor with improved depth functions, various depth modules and cameras and a software development kit (SDK).

The 'out-of-the-box kits', for example, provide imaging and vision engineers with ready-to-use components with which they can make applications 3D-capable. This minimizes long development cycles and significantly shortens time-to-market. One example: the RealSense depth cameras from Intel's 'D400' series can be added to existing prototypes via USB. The image resolution, high frame rates and integrated infrared projector support applications for indoor and outdoor environments where typical 3D solutions have previously failed. Depth data is generated by the integrated processor and can be forwarded directly from the camera to the selected evaluation platform. The 'RealSense SDK', which is available for Windows OS and Linux among others, includes sample codes, de-bugging tools and evaluation programs to accelerate vision projects.

The products in the RealSense line differ in specifications such as IR projector, field of view and shutter type. If deeper integration is required, the Intel vision processors can be integrated with customer-specific cameras and boards. RealSense ASICs calculate high-resolution 3D depth maps based on raw image streams from multiple cameras without the need for a dedicated graphics or host processor. This integrated solution can significantly reduce the load on the system's host processor.

Author:
Dr. Christopher Scheubel is responsible for IP and Business Development at Framos in Taufkirchen, Germany