How LPVIZ Augments Driving Reality
Going beyond a simple screen replacement, LPVIZ is an augmented reality driving assistance solution for the car. It allows displaying related content to a driver or passenger in 3D, superimposed to reality. Content can be placed anywhere inside the car, such as a virtual speedometer over the dashboard, and anywhere outside of the car, such as point-of-interest markers or navigation guidance.
The video on top of this post shows what a drive around the block in Azabujuban, Tokyo with LPVIZ looks like. A virtual dashboard is projected onto the center console of the vehicle. Arrows on the ground show lane guidance to the driver. Red Google Maps-style markers show points of interest. The virtual dashboard stays fixed to the same location in the car, even when the vehicle turns. The navigation arrows move smoothly and the point-of-interest markers are globally anchored.
Perfectly Tuned Components
LPVIZ consists of several components that all have to interact perfectly to create a compelling and safe augmentation experience. The below illustration shows a block diagram of how the hardware components are connected.
Accurate tracking is required to display useful content to the driver: the HMD pose in the local car coordinate system and the vehicle pose in a globally anchored frame. Precise calibration of all components of the solution is essential to provide the highest visual fidelity and driver safety. Our LPVIZ product makes all parts of the system available in a compact form factor, ready to be integrated with any vehicle.
The Past, Present and the Future
In the current development stage we’re focusing on the most essential aspects of the solution: displaying a virtual dashboard, navigation information and points-of-interest. While this is our proprietary content, we’re opening our software to work with 3rd party developers to create their own content building on our platform.
Currently we’re offering LPVIZ as a B2B solution for prototyping, design and research. However, we’re working on reducing system complexity to make it work as a consumer facing automotive after-market solution to be released later this year.
Towards a Consumer Product
We are very proud of the progress our team has made in the past months. We’re moving closer to making our vision of an augmented reality driving assistance system a reality for everyone. One very important take-away from our recent developments is that it’s indeed possible to provide real utility to the driver using technology that is readily available. It might still be early days, but we’re edging towards a product that could appeal to a wider consumer market. This is just the beginning.
In-Flight VR
Imagine soaring through the skies as a pilot, testing the limits of a helicopter’s capabilities while feeling the rush of wind and turbulence. Now imagine that you don’t see the real world outside and the safe landing pad that your helicopter is approaching but a virtual reality (VR) scene where you are homing in on a ship in high seas. The National Research Council Canada (NRC) and Defence Research and Development Canada (DRDC) have brought this experience to life with their groundbreaking Integrated Reality In-Flight Simulation (IRIS).
IRIS is not your ordinary simulator; for one, it’s not sitting on a hexapod, it’s airborne. It’s a variable-stability helicopter based on the Bell 412 that can behave like other aircraft and can simulate varying weather conditions; combine that with a VR environment and you have a tool that allows safely training operations in the most adverse conditions. In particular it is used for Ship Helicopter Operating Limitations (SHOL) testing.
Mission-Critical Application with LPVR-DUO
The LPVR-DUO system is what makes VR possible on this constantly moving platform. This cutting-edge AR/VR tracking system seamlessly merges the inertial measurements taken by the headset with the helicopter’s motion data and a camera system mounted inside the cabin to provide the correct visuals to the pilot. The challenges of using cameras to track the VR headset inside the tight environment of the helicopter while lighting conditions are ever-changing are overcome by using an ART SmartTrack 3 system. This system follows an arrangement of reflective markers attached to the pilot’s helmet. The VR headset is attached to the helmet in such a way that the pilot can wear it as if it were a pair of night vision goggles. Put together, this allows displaying a virtual world to the pilot, even in the most extreme maneuvers.
To ensure an authentic experience, the IRIS system incorporates real-time turbulence models, meticulously crafted from wind tunnel trials. These turbulence effects are seamlessly integrated into the aircraft’s motion and into the VR scene, providing pilots with precise proprioceptive and vestibular cues. It’s a symphony of technology and innovation in the world of aviation testing.
In-Cockpit Implementation
The optical tracking system relies on highly reflective marker targets on the helmet to track movement in three dimensions. Initially, only five markers were installed, strategically placed for optimal tracking. But the pursuit of perfection led NRC to create custom 3D-printed low-reflectivity helmet molds, allowing them to mount a dozen small passive markers. This significantly improved tracking reliability in various lighting conditions and allowed for a wider range of head movement.
Recently, NRC put this remarkable concept to the test with actual flight trials. The response from pilots was nothing short of exhilarating. They found the system required minimal adaptation, exhibited no noticeable lag, and, perhaps most impressively, didn’t induce any motion sickness. Even the turbulence effects felt incredibly realistic. Surprisingly, the typical VR drawbacks, such as resolution and field of view limitations, had minimal impact, especially during close-in shipboard operations. It’s safe to say that IRIS has set a new standard for effective and immersive aviation testing.
Publication of Results
The NRC team presented their results at the Vertical Flight Society’s 79th Annual Form in two papers [1] and [2] and they also have a blog post on their site.
NOTE: Image contents courtesy of Aerospace Research Centre, National Research Council of Canada (NRC) – Ottawa, ON, Canada
Varjo Releases Mixed Reality Headset XR-4
A leap forward in XR comes from our partners at Varjo, who have been pushing what is possible in VR and AR for the past few years.
Today, they announced their new flagship headset, the XR-4! We have worked with them to ensure that our LPVR software series is ready for it from the start.
The XR-4 features unmatched visual fidelity not only of VR content (expanded to a field-of-view of 120×105 deg), but also of mixed-reality video pass-through, reaching a pixel density of 51ppd in the central viewing area. Light sensors further improve the quality of immersion by adjusting the image display to external lighting conditions.
LPVR Support is Ready
All this is great, but what is a headset without a world to immerse in? What is a VR racetrack without feeling the real motions of the car? Varjo clearly understood and prepared, so they collaborated with us to get LPVR ready for the launch of the XR-4.
We have everything prepared for you!
- Use your existing camera systems and props to augment the virtual world.
- Use the industrial-grade precision of ART, OptiTrack, and Vicon tracking systems with the Varjo XR-4 and our custom marker holders.
- Integrate the HMD with your race-car simulator or fighter jet platform.
- Do all of this with the XR-4. LPVR-CAD, and LPVR-DUO will make sure that you are tracked perfectly.
*We offer full solutions of state-of-the-art tracking systems and content using the Varjo XR-4.
For specific supported HMD models, please contact us for detailed information.
