Perspectives

AI-Boosted VR/AR Could Transform Training

Contributed by Nirmal Mehta, Drew Farris, and Cameron Kruse

Artificial intelligence & immersive merge for more realistic simulations

Until recently, the reality of virtual reality (VR) was kind of disappointing. Popular depictions like The Matrix and Star Trek’s Holodeck stoked the imagination with simulations that were discernible from the real world only by the fact that real-world rules didn’t apply—anything was possible. The actual technology paled by comparison. Cost-prohibitive equipment, unconvincing graphics, clumsy and limited interactivity, and the fact that it nauseated many users added up to limit VR’s appeal.

But recent developments have lessened the gap between the VR of sci-fi and the actual technology, and rapid advances in artificial intelligence (AI) and machine learning (ML) promise to close it even further. New headsets like the HTC VIVE and the Oculus Rift are widely affordable, and increasingly powerful processors have enabled more convincingly rendered worlds with faster frame rates that reduce motion sickness. This has made VR more popular among consumers, and dramatically increased its utility for education and training.

Applied to the development of VR and augmented reality (AR)—known together as immersive technology—increasingly powerful machine learning techniques will enable even more detailed and interactive worlds with more realistic inhabitants and more intuitive, less distracting user experiences (UX). These improvements will profoundly expand not only immersive’s popular appeal, but also its value as an education and training tool.

Control Without Controllers

Consider UX. Now, users interact with simulated environments primarily through hardware like keyboards and handheld controllers that are external to the simulation, preventing feelings of full immersion. When motion-based controls do come into play, they’re clumsy and imprecise.

Machine learning-powered advances in computer vision and spatial awareness will enable enhanced motion tracking, allowing designers to create experiences that users control with subtle, natural movements of their fingers and hands. Advances in natural language processing—a computer’s ability to understand and respond to free-flowing spoken and written words—will enable intuitive voice commands and character interactions, making for even more immersive experiences.

For purposes of entertainment, a user interface that’s seamlessly integrated into the simulated experience through movement and voice means users will feel more fully transported into the world that designers have created for them. For training purposes, users will feel more as if they’re actually present in the scenario at hand, learning to make decisions in the moment, under pressure that seems real.

Capabilities

Immersive Experience

By immersive, we mean the presentation of artificial environments that replicate the real world. It’s a dynamic mix of virtual reality, augmented reality, mixed reality, and enhanced input devices. Read More

Capabilities

Characters That Learn

Advances in AI will increase that perceived realism even further through improvements to the characters that users interact with in immersive simulations. In the past, characters’ actions had to be preprogrammed as reactions to predicted user behaviors. Now, using deep learning—a subfield of machine learning that employs artificial neural networks inspired by the structure and function of the human brain—computer scientists can create complex algorithmic character controls that learn from user behaviors, allowing for virtual colleagues and adversaries that behave in less predictable, more human-like ways. For something like a combat simulation, users will walk away feeling much more like they’ve actually experienced the uncertain immediacies of the battlefield.

Automated World Building

AI breakthroughs could also make the development of realistic simulated environments cheaper and less time consuming. Today, gaming studios expend more than half of their resources employing artists to render characters and environments by hand. This process could be sped up and made more cost efficient by using deep learning-created algorithms to automate the generation of landscapes and animation of characters. For agencies and businesses, that could mean the ability to commission more numerous and realistic training simulations without busting budgets.

Whether you’re interested in immersive technology as an incredibly cool form of transportive entertainment, or as a powerful training tool that improves safety and effectiveness in the workplace and on the battlefield, AI-fueled advancements in the field should have you excited.

For more insights into the AI/Immersive convergence, listen to our podcast on the subject.