FlightSafety Mixes Reality in New Simulator
by Matt Thurber
An interesting development that might have a profound effect on the training simulation industry is underway at FlightSafety International’s Visual Systems division in St. Louis, Missouri. Some lucky visitors to the Heli-Expo show earlier this year in Anaheim, California, were able to get a private demo of the new technology, which FlightSafety calls a “mixed-reality” flight simulator.
The new simulator takes advantage of the many years of development of FlightSafety’s Vital image generator and visual display, marrying the most recent version with virtual reality headwear and hardware controls to create a mixed-reality simulator.
Virtual reality or VR—in the form of eyewear that makes the user feel immersed in the simulation—is nothing new in flight simulators, especially in consumer types. But a problem with these setups is that there is no good way for users to experience moving the simulated aircraft’s controls within the VR environment. There are controls that a pilot can use to simulate pushing the aircraft’s buttons and turning its knobs, but there is no feedback mechanism for feeling the action of pressing and turning. And such VR setups don’t allow the user to look outside the VR environment to manipulate physical controls.
FlightSafety’s mixed-reality simulator solves that problem by combining special VR gear with a hardware platform that replicates the cockpit controls and enables the realistic feel of actuating avionics and other controls. The VR headset is fitted with external cameras, and this allows FlightSafety engineers to program what the pilot sees in virtual reality and what real-world items are passed through, such as avionics.
For example, when the pilots look out the windows, they can see a virtual rendering of the world, just the same as they would see out the windows in a FlightSafety full flight simulator. In the helicopter, VR adds extra views, such as the ability to look down and see the helicopter’s skids as they fly over terrain or water or to look back into the rear of the helicopter and see empty seats and the view out the rear windows. But then if pilots need to change a frequency, program a flight plan, or change some setting on the instrument panel, they would see these through the camera view and be able to reach out and touch the associated physical controls. For control of the aircraft itself, pilots fly with a familiar yoke for fixed-wing airplanes or cyclic and collective for rotorcraft.
The mixed-reality trainer comes in three types: a desktop device, a standard version, and a pro version. Each is powered by FlightSafety’s own aerodynamic modeling and simulation software, so the experience is consistent for each trainer type and also when flying in a FlightSafety fixed-base or full-motion simulator, which all run the Vital visual system.
The standard version is fairly light and easy to move around. It consists of a powerful, rugged PC, the head-mounted display, the Vital display, and controls including a cyclic and collective. The other two versions include the light, which is basically a laptop PC that can run the software along with a yoke control and the headset. The high-end pro version, which was demoed at Heli-Expo, adds a motion base to the standard version and a force feedback system for the flight controls to provide a more realistic control feel. The pro version can be moved around on a set of wheels, although it isn’t as convenient to transport.
With the light version, said FlightSafety visual database product manager Mike Johnson, “We can start a level of training." The standard version "still supports a huge amount of training value." The idea is that a client could use the light version for training at home, including preparing for classes at a FlightSafety learning center. But beyond that, the standard and pro versions could deliver a lot of training value at a far lower cost than a full-flight simulator or even a traditional fixed-based training device.
“We’re definitely in a phase of heavy investment in technology,” Johnson said. “We love to tap into the technology and come up with simulation solutions that support that.” And as it turns out, the timing of the coronavirus pandemic highlights the utility of the mixed-reality simulation. With travel greatly restricted, an operator could bring a trainer to its facility to help pilots stay current. A military customer might want to keep one on ships to help pilots make more efficient use of downtime. “It fits anywhere you can plug in a lamp,” he said. “It’s so flexible and lightweight, it could be a direct use-case for the pandemic.”
Vital Visuals
FlightSafety’s continued refinement of the Vital visual display means that mixed-reality users get to experience the latest technology, but another important element is the simulation of the various sensors that feed information to pilots. This includes radar and enhanced vision systems. “They’re all perfectly correlated,” said James Wheeler, general manager of the Visual Systems division. “We want it to be exactly the same as what you see in [the cockpit]. It’s a valuable detail for us.”
The same is true for FlightSafety’s worldwide database over more than 600 airports and heliports, available to be used on any training platform. Plus FlightSafety has developed specialized geo-specific training areas including oil platforms, military facilities, first-responder scenarios, and hospital helipads.
The geo-specific aspect not only lets pilots see accurate depictions of airport features, buildings, etc., but it also means that flying into a fixed feature results in a crash, just like in the real world. “When you see a building in the virtual environment, there is a building there,” Wheeler said. “For FlightSafety, investing in a geo-specific database that is correct pays big dividends. You truly train the way you fly.”
Mixed-reality Mission
FlightSafety isn’t trying to replace its core simulator products with mixed-reality devices, according to Wheeler. “[Full-flight simulators] are 100 percent necessary and will always have a place,” he said. That said, the mixed-reality simulator supplements traditional training. For example, there may be pilots in a class who need more practice in flying the aircraft before going into the full-flight simulator. “They can learn adaptively,” he said, “and we can track where they are proficient and where they aren’t and focus on the areas where they are not proficient. We can give them a lot more repetitions.”
The new FlightSmart training evaluation tools that FlightSafety has developed would fit perfectly into use of mixed-reality training, by informing instructors about students’ weak areas. “This [mixed-reality] integrated with FlightSmart really starts an empowered learning ecosystem,” he said, “an adaptive learning system.”
FlightSafety has tested the mixed-reality training system at its training academy in Vero Beach, Florida, for ab initio student pilots. The results were clearly positive, with average solo time decreasing to 31.6 hours from 39.9, average number of repeated lessons dropping to 6.1 from 14.3, and average training days to solo down to 90 versus 175. Similar results were seen during a trial at Sheppard Air Force Base in T-6A single-engine turboprop training airplanes.
The big benefit of the mixed-reality simulators for early pilot training is that students can fly traffic patterns and visual maneuvers with much better fidelity compared to a desktop simulator with a limited field of view. And the students can practice on their own, too. Military pilots could fly in formation and practice various other missions, such as flying into an unfamiliar airport, emergency procedures, and more. New pilot screening is another opportunity.
Mixed-reality simulator testing is ongoing and includes work with the U.S. Air Force’s 9th Reconnaissance Wing at Beale AFB in California for T-38A jet trainers, Air Force pilot screening, trials with European NATO members for jet pilot training, and ab initio pilot testing with Purdue University. For the Sheppard AFB program, FlightSafety incorporated PilotEdge’s real-time live air traffic control service.
Another opportunity for the mixed-reality simulator is remote training, where the instructor and student are in different locations.
FlightSafety is working on gaining approval from regulatory authorities to allow training in the mixed-reality simulator to count towards training requirements. “We are eager to work with authorities with this type of technology and how it should be accredited,” said Johnson. “I think we’ll see a tipping point on the civil side once we gain accreditation. But even before gaining credit, there’s still a lot of value.”
Source : AINONLINE