Creating Real-time Digital Twins with Intel RealSense Depth Cameras

Developers inspire our work, we’re constantly amazed at the innovation their solutions. One of these such examples is by Spectacular AI using Intel® RealSense™ Depth Cameras to create digital twins in real-time.

The Spectacular AI SDK provides real-time inside-out tracking capabilities tailored for the Intel RealSense Depth Camera D455. Utilizing Visual Inertial Odometry (VIO), it fuses visual data from the camera with readings from internal sensors (accelerometer and gyroscope) to accurately determine the device’s position, orientation, and movement. This integration ensures robust tracking performance with minimal latency, eliminating the need for external positioning systems. The solution is optimized for low-powered embedded devices, catering to diverse applications such as augmented reality, 3D reconstructions, autonomous drones, and robotics.

One such application is the popular Gaussian Splatting, which is a new way of creating 3D scenes from 2D images. It works by projecting the images into a 3D space as small “blobs”, called splats, that have shape like a bell curve, hence the name Gaussian. To do this, camera poses and point clouds corresponding to each image are required, and these are provided by our SDK. Together with this information, software such as NeRF Studio can be used to train the splats to match the 2D images as closely as possible. Once the training is done, the result is a 3D model that can be freely rotated around, as seen in the video. The rendering is fast enough to be done even on mobile devices, and additionally provides more realistic shading and lighting than traditional mesh-based reconstructions.

Spectacular AI is an Aalto University start-up from Helsinki, founded in 2021. Their team is a unique mixture of seasoned software professionals plus established computer vision and machine learning researchers. They solve challenging problems related to Computer Vision, Spatial AI, Sensor Fusion and SLAM (Simultaneous Localization and Mapping).

If you are interested in learning more about this cool digital twin technology, checkout their website and their Intel RealSense GitHub repository for more information.

If you are a developer building novel technologies with Intel RealSense cameras, we would love to hear from you! Send us an email at .