The holy grail of virtual reality may be a training reality for future troops.
The military’s increased reliance on virtual reality to train warfighters may be converging with rapid advances in technology that will bring the holodeck of Star Trek fame closer to actuality. Holograms and similar technologies offer the possibility of realistic, cost-effective training and education for a broad array of military missions and commercial applications.
The U.S. military already has put virtual humans to good use. For example, the Institute for Creative Technologies (ICT) at the University of Southern California, Los Angeles, has used virtual reality characters to touch warfighters in one way or another before, during and after combat deployments. The Army Research Laboratory funds the institute, which was initiated with the holodeck in mind. The ICT has created several programs to train soldiers in situational awareness and for urban combat and counterimprovised explosive device missions. The institute’s virtual humans also help counsel warfighters suffering from post-traumatic stress disorder.
Maj. Gen. Jonathan Maddux, USA, program executive officer for simulation, training and instrumentation, indicates that hologram technology “continues to be a work in progress” but that Tactical Digital Hologram technology has “shown promise” with the U.S. Army Special Forces in Afghanistan and Iraq. Special operators use the technology to create 3-D maps of villages or specific buildings.
The ICT’s proven technologies are sometimes referred to as holograms, but scientifically speaking, they are not true holograms. Experts agree that the term “hologram” has come to mean different things to different people. Alkiviades “Alki” David, CEO, HologramUSA, Beverly Hills, California, explains that current technologies do not produce true 3-D holograms but instead often rely on two-dimensional images made to look three-dimensional.
Still, the ICT and others are pushing the state of the art. In fact, David says his company, which hopes to break into the military market, will soon make good on ahead-of-its-time patented technology and unveil a true hologram in the coming months. “We’ve recently licensed a patented technology that projects the image into the audience or into the space, so you can actually put your hands through the light when you project the image. It’s actually a bona fide hologram,” David reveals. He adds that the patent is 10 years old, but recent breakthroughs now make the technology possible.
The company has not yet come up with a name for the product and has not yet built a prototype. “The first model is going to cost about $6 million or $7 million to develop because we have to build tools to build the lenses. We need a 20-foot lens, a Hubble telescope-size lens, which in itself was fraught with problems,” David reports. “We’re going to launch a commercial version of the prototype in maybe six to eight months.”
David says the company has held talks with the Army, but he is not certain how much he is allowed to reveal. Among other accomplishments, his company projected 3-D images of Narendra Modi around India during Modi’s campaign for prime minster, which he ultimately won. “Our telepresence technology can be used for various military and defense uses toward communication, but it can also be used for simulation as well as modeling. The applications are varied, but it could be extremely useful to the military,” David asserts.
Scalability is one of the key advantages of holograms, he says. For example, if the Army needs to model a weapon system under development, it could create a giant hologram and see the “inner mechanical workings” of a prototypical system. That would allow the service to make changes before building a prototype, ultimately saving time and money. The company already has created a car configuration application for Audi that allows people to select the color and other features of a car. “And we can certainly do the same for military vehicles or weaponry,” David adds.
A giant hologram also will offer perspective over long distances. “If you wanted to watch the impact of shooting somebody over a large distance and watch the trajectory, you could,” David asserts. “If you want to run 100 meters against Usain Bolt, you could.” Presently, HologramUSA is in talks with a defense company to develop a flight simulator. It also has been approached by an education oversight organization in California about creating characters for in-classroom instruction.
The company does not yet have military customers, but it does offer military products. “We’ve already developed a marvelous motion-capture technique, which you can use for simulating urban combat. It’s a shooter application. We’ve been projecting our various characters using electronic weapons to do urban combat training. It’s pretty cool,” David asserts. The program offers 16 to 18 characters, including men, women and children, both service members and civilians. The environments also are interchangeable.
Todd Richmond, ICT director of advanced prototypes, agrees that holograms and hologram-type technologies offer a broad range of training potential. “It’s all over the map. We’ve used virtual characters for a whole bunch of different tasks,” Richmond says.
He posits that a holographic virtual human could follow a service member throughout the person’s career. We’ve all had teachers, coaches or mentors who have significantly influenced our lives, Richmond says, and holograms could do the same. “That virtual character, if it’s tied into the right databases, knows the classes you’ve taken, has been around you enough to know your strengths and weaknesses, and [can] guide you through challenges and experiences that you come upon as you advance through your career,” Richmond suggests.
Advanced artificial intelligence technology would be needed for virtual life coaches, and the technology is making rapid strides. “Artificial intelligence ends up being a big part of it, and it is a very broad field with a difficult set of challenges. That’s why we have people working on the basic research side of things, and we also have people building the advanced prototypes so that we can actually put these things out in the field and test them to see what breaks,” Richmond says.
The level of artificial intelligence needed depends largely on the purpose the virtual human serves. “It runs the gamut from not-very-smart artificial intelligence to smart within a very limited domain to systems that are broader and have broader simulations running underneath them,” Richmond elaborates.
Both David and Richmond cite the affordability of virtual characters as a major benefit. Live role players are used at a variety of military training sites, including the Infantry Immersion Trainer at Camp Pendleton, the Army’s National Training Center at Fort Irwin, California, and various urban combat training areas. “Human role players are really expensive over time. If you create a virtual version, that character can run 24 hours a day, seven days a week. It doesn’t have to take bathroom breaks. It doesn’t have to stop to eat. It doesn’t risk injury. You can damage the equipment, but you’re not putting humans at risk,” Richmond observes.
Furthermore, virtual characters perform consistently. “One problem with humans is that there are good role players and there are not-so-good role players,” Richmond offers.
Other recent breakthroughs in holograms include improvements to the light-emitting diodes (LEDs) needed to project the images. In short, the distance between the LED lights has decreased from 5 millimeters to 1 millimeter. While the change may seem simple to the uninitiated, David says it allows holograms and their near-cousin technologies to accomplish tasks never done before. “The use of LED technology has been refined radically. The brightness now allows us to project these images in bright sunlight without it needing a dark space behind it. That’s a huge advance. You no longer need to be in a darkened environment, and you no longer need to worry about lights coming in and ruining the image because the light of the LED is brighter than sunlight,” David explains, adding that the company’s soon-to-be-released product will take advantage of that and other technological gains.
Meanwhile, ICT research and development includes projector array technology. Researchers have used the Microsoft Kinect for one version of a projector array; for another, they set up 100 cellphone cameras and 100 Raspberry Pis, which are credit card-sized single-board computers. “We’re increasingly exploring these projector arrays and also head-mounted displays. We’ve used things like the Oculus Rift in a head-mounted display, and we’ve used light-field technology, which is the basis of a hologram, but we then rendered that in the head-mounted field,” Richmond reports. He adds that the technique recently won a virtual reality award at the premier graphics and simulation conference known as SIGGRAPH.
While cost and logistics still can be an issue with large projector arrays, Richmond says the commercial world is continually improving 3-D imaging hardware. “That problem will get solved over time. The technology’s getting cheaper almost month by month. The tech sector is really commoditizing the hardware pretty quickly,” Richmond states.
In other advances, Microsoft is soon expected to release its HoloLens, which the company markets as the first fully untethered, holographic computer, enabling high-definition holograms to integrate with the user’s world. In the frequently asked questions section of its website, Microsoft says holographic computing experiences with the HoloLens are different from existing techniques. With augmented reality, the user sees a layer or screen of data that overlays the real world. With virtual reality, the user is completely immersed in a computer-generated world. The HoloLens delivers a mixed reality that lets users enjoy digital life while staying more connected to the world around them, according to Microsoft, which did not respond to a request for an interview.
Regardless of the technological advances, holograms and hologram-lite technologies always will have one limitation. “Ultimately, we are limited only by our own imaginations,” David says.
Reprinted from SIGNAL Online, Dec 2015 with permission of Signal Magazine. Copyright 2015. All rights reserved.