Brand new feature for Intel® RealSense™ Stereo Depth Cameras – Self Calibration

Stereo depth cameras like the Intel® RealSense™ D400 series are based on calculating depth from stereo vision – that is, two sensors separated by a known baseline. Those two sensors or imagers are built to be very sturdy, with laser-fused steel cages designed to help maintain calibration and performance over their lifetime. This is extremely important, since performance is dependent on the known spacing between the two cameras; the algorithm that calculates depth for every pixel needs that specific piece of information to accurately understand a scene.

Innovative industry leading feature

While the cameras are designed to maintain this calibration throughout their lifetime, there are some conditions which can lead to degradation of the calibration over time, such as exposure to extreme temperature cycling, excessive shock or vibration. These can lead to two different distortions – extrinsic distortions which relate to the microscopic bending of the stiffener on which the sensors are mounted, and intrinsic distortions which are caused by microscopic shifts in the sensor lens positions.

Comparisons of the Point Cloud of a well calibrated camera (LEFT) with a degraded camera (RIGHT) for a flat textured wall. The lower bumpiness on the left is preferred.

The new Intel RealSense on-chip calibration allows you to run an exact health check on the system, to make sure the sensors are performing in the optimal range. This number is a complete indication of the calibration health of the device and can be tracked over time. The Self-Calibration feature also allows you to run a re-calibration as quickly as 0.6 seconds – the camera depth can be recovered to near-perfect performance in less than a second. With no need for targets, checker pattern walls or complex setup like specific motion paths for the camera or the need for the camera to be static, self-calibration is extremely simple, just call an on-chip function.

A set of scenes that have successfully been used for self-calibration, ranging from the ideal flat textured target without projector illumination (A), to scenes with projector on (B, D, E, G), to outdoor scenes in bright sunlight (F, I).

All these features are now embedded into the updated firmware on board the camera’s vision processor, without needing compute from whatever host platform the camera is attached to. If the health check process determines that the new calibration data is superior to the old, the updated and improved calibration data can then be burned directly to the flash memory, for improved performance moving forward. This applies to all existing cameras in the D400 series – just update your firmware following the instructions in the Intel® RealSense™ Documentation.

Read the whitepaper here

See it in action