Three Scanning Applications with ImFusion

Intelâ„¢ RealSense® Depth Cameras have a wide variety of use cases and applications, one of which is high quality real-time surface scanning. ImFusion, a Munich based company focused on computer vision and medical imaging solutions have recently added support for Intelâ„¢ RealSense® D400 series into its product lineup of 3D scanning and surface processing software. Their software solutions and software development kits (SDK) are designed so that customers can quickly prototype and build their own value-added solutions on top of ImFusion’s technology. To better explain the possibilities and applications here, we are going to share three different case studies using both the Intelâ„¢ RealSense® Depth Cameras and ImFusion’s applications.

Multi-Sensor 3D body scanning

One of the applications ImFusion provides is RecFusion Pro, a dedicated 3D scanning application targeted at end-users. Using this technology, multiple Intelâ„¢ RealSense® Depth Cameras can be combined into one scanning system for performing fast, turntable-based scans of people. The ideal setup here is to use three to five D435 or D415 sensors oriented vertically on top of each other, fixed in front of the turntable. The data captured by the sensors is used to reconstruct the 3D shape of the person in real time within one turntable revolution. After the scan is complete, further processing and re-texturing improves the accuracy and color of the model. The highly detailed resulting mesh is then usable for a variety of applications, such as 3D printing or body measurement.

From left to right – color and depth information from each of the three cameras, colored scan, shaded scan.

Face Reconstruction

Using the ImFusion SDK, as an example of how their Suite accelerates depth application development, ImFusion developed a 3D face scanning application. It uses a small number of views from an Intelâ„¢ RealSense® Depth Camera D415 and combines them together into a textured 3D surface model. The ImFusion SDK allows you to easily acquire the raw depth and color data from the camera, filter and pre-process to remove noise, outlier points and unnecessary background information and then take the resulting pointclouds, combine and align them together. The final output is a textured mesh representation of the face which can be used in many different ways – for example, accurately aligned virtual makeup testing, or effective and natural looking hair style experiments (before you cut!).

From left to right: Color image, point cloud with normal coloring, reconstructed and textured face mesh.

Surface to CT scan alignment

When medical professionals are preparing highly complex surgeries, such as facial reconstruction, or surgeries for bone injury, they often perform a CT (Computed Tomography) scan. These scans can sometimes happen days in advance of a surgery. By using an Intelâ„¢ RealSense® Depth camera combined with the ImFusion SDK support for a wide range of medical and industrial volumetric scan data, a depth image can be aligned to the CT scan in order to use the pre-operative planning done on the CT for guidance during the surgery. Another application would be to find any discrepancies between scans – that is, compare whether something has changed from the CT scan to the day of surgery.

In this example, ImFusion scanned a 3D printed skull for which a corresponding CT scan was available. The depth data from the Intelâ„¢ RealSense® Depth Camera was aligned directly to the skull surface seen in the CT scan, allowing for direct comparisons. Imagine a spinal surgeon being able to take a live surface scan during surgery to exactly, precisely and minutely guide the procedure based on a pre-operative plan they previously designed based on the CT scan.

Top left: Scanned point cloud with normal coloring,
Top right: Mesh obtained from point cloud,
Bottom left: CT scan with volume rendering and cut planes
Bottom right: Scanned surface in purple registered to CT scan.

Just the beginning

These three presented applications highlight just a few of the many possible solutions which can be built by combining the Intelâ„¢ RealSense® Depth Cameras with the software development tools provided by ImFusion. For more information about ImFusion products and the services they offer, please visit their website.