Meet the AI museum guide who can screen for COVID-19 (Updated May 2021)

The Ontario Regiment Museum contains the largest collection of operational historical military vehicles in North America, some of which are loaned out for TV and film productions, reenactments, and parades. To maintain these living historical artifacts, they must be regularly driven and repaired by a small army of volunteers, who continue to support the museum throughout the pandemic.

One of the newer ‘volunteers’ to support the museum is Master Corporal Lana, an artificially intelligent character who can welcome visitors, check people in and provide useful information such as directions or event specific messages. She can perform a number of functions such as counting visitors and providing crowd control or checking tickets. Master Corporal Lana is based on the Animated Virtual Agent (AVA™) designed by CloudConstable as a touchless virtual concierge or greeting host.

The technology

MCpl Lana is designed to allow for natural interaction and incorporates a number of Intel® technologies to help make that happen. An Intel NUC 9 Pro mini PC provides the performance the system needs to operate as a smart kiosk and Intel vPro® technology enables secure remote management. Natural language processing is used to allow Lana to hold a conversation and both ask and answer questions from visitors and volunteers, and the Intel distribution of the OpenVINO™ toolkit helps with AI inference acceleration.

An Intel RealSense depth camera D415 is an important part of the solution – if MCpl Lana asks a question, she can both recognize a verbal answer or a non-verbal nod or headshake. Gesture controls like this allow the solution to be touch free while feeling natural. In addition to these gestures, the depth camera assists with authenticating registered users like the 100 plus volunteers who help maintain the fleet of historical military vehicles. These are the people that make sure the fuel doesn’t go bad, keep vehicle seals in good shape, and keep impossible-to-replace parts from breaking down. They drive tanks in a nearby field, keeping them operational so history buffs can experience them live, or see them on TV.

The Intel RealSense depth camera is also used to enable a kind of ‘virtual privacy shield.’ The system has a privacy limit parameter, usually set to about 2 m. This is done so that only when a subject approaches Lana’s station within the defined distance does Lana begin to interact with the subject — which starts by trying to recognize the subject’s face so she can greet the subject by name. In optimal conditions the system could recognize a subject who has previously opted in from a distance of 4 or 5 meters away, but by using the depth camera to set a privacy limit, passers by are ignored and Lana has a more natural conversation only with those users who approach her.

Safety screening

With so many volunteers and an important reason to require them to still be able to enter the museum, health became MCpl Lana’s new priority. CloudConstable worked with The Ontario Regiment Museum to take Lana from her initially designated role as a new way to engage with visitors and expand her role to COVID-19 safety screening.

MCpl Lana, a customized animated virtual assistant, screens Ontario Regiment (RCAC) Museum volunteers for COVID-19 with temperature scanning and questions.

Now, MCpl Lana is in a new vestibule by the employee entrance. Volunteers can check in and then answer questions about their health and possible contact with the virus. The Intel RealSense depth camera, as well as assisting with the gesture control in the system is part of a new custom sensor array that also includes a touchless LWIR thermal scanner. The two sensors together help to detect potentially elevated body temperatures. Since the reading from the thermal scanner can be affected by how far the person is from the sensor, calibrating the thermal sensor and the depth camera together ensures more accurate temperature readings, where higher temperatures could be a warning sign of potential COVID-19 or flu.

Users that present a risk to others based on their question answers and/or temperature reading can then be sent home to rest. And since the system is checking the volunteers in and out, in the eventuality that a volunteer does become infected, there’s a record of potential contacts who were in the building at the same time.

Next steps

MCpl Lana is just one character made possible by the AVA™ system. As an animated character, it is completely customizable in terms of language, gender, culture and branding. AVA can give custom greetings and visitor instructions, as well as answer location specific questions. CloudConstable anticipates that these customizable AI assistants can help other high-foot-traffic environments like retail or healthcare locations. Using an AVA as a concierge or check-in can give businesses a way to increase customer safety without putting a human employee at risk, but still give visitors a more human way to interact with the technology keeping them safe.

Jeremy Blowers, the Executive Director of The Ontario Regiment Museum, was impressed to see that people of all backgrounds and ages were intrigued by MCpl Lana, and most were eager to sign up for her facial authentication functionality.

“Obviously MCpl Lana is great because she doesn’t need to take breaks, never has a bad day, and is always on message,” he said. “But it was fantastic to see the level of excitement about her. This moment could be a leap forward for this technology.”