Making Magic – Art and technology working together

At Intel® RealSense™ Technology, one of the things we like to share is the innovative use of our technology for large scale interactive installations. There are many uses for depth and tracking technologies that are entirely practical, from every day enterprise solutions, to industrial robotics and drones. There are also the standalone pieces of magic, when developers use our technology to power an experience people have never seen before.

For Nuit Blanche Toronto 2018, production studio Array of Stars was tapped by ad agency UNION to produce a unique interactive piece to highlight the INFINITI QX50. The goal of the installation was to allow people to see themselves inside the car, as well as highlight a few key features. In order to do this, the SUV was wrapped in 50,000 individual LED lights, with 6 Intel RealSense D415 depth cameras positioned around the vehicle to feed environmental input into the LED system.

From the standpoint of the experience, attendees to the conference would be able to move a full 360° around the car and in real time see themselves reflected in the LEDs. As computer vision hardware becomes more and more mainstream, we will start to see more experiences like this where interaction with digital computers and displays becomes more intuitive. Instead of needing to click a mouse or tap on a touchscreen, depth cameras allow developers to build experiences without the need for such affordances.

In this case, anyone standing in front of the car did not have to try and figure out how to interact with the car – when we see ourselves and our motion reflected with very low latency, we can immediately intuitively understand what to do. Once we have that understanding, our natural instinct is to start to play, to find out what else we can do. This behavior is rewarded in the experience too, with the color of the lights changing depending on how fast the person moved in front of the car, with red lights indicating fast motion, or green for someone standing still or moving slower. This was to highlight a particular feature of the car – a new kind of advanced variable compression engine, called the VC-Turbo.

Since the Intel RealSense Depth cameras operate at up to 90 frames per second, getting this active real time data was an easy feat. By feeding the depth information from the D415 directly into the Unity game engine via this Unity wrapper, different particle effects could be applied and then output to the LEDs, for a fun and eye-catching experience. This highly visual piece was experienced by thousands of people at the Nuit Blanche show, and has since toured Canada reaching many more.

Check out the video of this installation in action.