Depth and Gaming

Depth Cameras and Gaming

For many people, their first introduction to a depth camera was with the release of the original Microsoft Kinect. The Kinect was a depth camera that utilized coded or structured light depth sensing techniques to evaluate the scene in front of the camera, allowing players to interact with games without the need for a controller, mouse or keyboard. Later generations of Kinect utilize other depth sensing techniques for the same purpose. Intel RealSense depth cameras can be used in the same way as the Kinect for interactivity in gaming.

Fun and games

As we have previously discussed on this blog, when it comes to gesture control, skeletal tracking and hand tracking, the interaction methods you use should always be based on what kind of interactions you expect your players to control. Depth camera based games that work well are things like dance, rhythm or pose based games that require the player to make a specific shape with their body – something that can easily be tracked with skeletal tracking software like cubemos. Some sports style games also work well, for example golf, tennis or soccer, although some faster paced sports games could be challenging for the player. While skeletal tracking systems work well, there can be some latency between tracking the body and correctly displaying that input on screen (and allowing it to be used to control some aspect of the game).

When it comes to designing games that are fun, depth camera-based games have an advantage over controller-based games. People find motion inherently enjoyable, especially in our often very sedentary lives, an opportunity to get off the couch to play a game can be very appealing, as well as socially engaging for groups to play together. VR games like Beat Saber don’t utilize a depth camera, but part of the appeal of the game is the movement. The designers of the game wanted people to look and feel cool when they played the game and designed the scoring system appropriately. Hundreds of Youtube videos of people playing the game and looking like they are dancing Jedi also boosted its popularity significantly.

Other concepts that could work well as depth camera based games could be flying simulators – think less of a simulator where you are a pilot, and more a simulator where you are a small child, pretending to be a plane with your arms held wide.

Games with purpose

When you remove the need for a controller from a game, play can become more intuitive, and more accessible. As an example, Makaton is a gesture based language similar to sign language which is used to support spoken language for people with learning or communication difficulties. In the game below “Boris”, players learn Makaton signs to play the game and unlock different puzzles. A depth camera is used to identify the gestures as players make them and use them to control the character through a variety of tasks and puzzles.

The engagement of a game can be much more compelling and enjoyable as a learning experience, allowing faster progress with the learning of signs in a low friction and risk free environment.

In this study the authors designed a system to use a depth camera and a VR headset to create games that help prevent falls in the elderly. Again, linking the physical activities to gameplay improve engagement and encourages posture improvement, something critical to fall prevention.

Creating games using Intel RealSense Depth cameras.

While Intel RealSense depth cameras are not designed specifically as a gaming peripheral, using the Intel RealSense SDK 2.0 allows you to use any of the current generation of depth cameras within a game engine – a Unity wrapper is included in the SDK, as well as some support for Unreal Engine. Alternately, a number of skeletal and gesture tracking systems like the previously mentioned cubemos support Intel RealSense depth cameras and can be used as inputs for gaming.

Have you created a game using Intel RealSense depth cameras? Tag us on social media @intelrealsense and show us what you’ve made.