About Luis Peralta

Image Processing, CV, and SLAM: Because reality is just a very high-resolution, poorly-calibrated, and noisy video feed.

LPVISION-HeadPose Tracker: Simplifying Head Tracking for AI-Driven Spatial Computing

Recently, at LP-Research, we have been working to simplify AI-driven spatial computing through a reliable, markerless head-tracking solution. The LPVISION-HeadPose Tracker is the result: an advanced 6DOF system that uses a standard RGB camera and advanced AI. By doing so, it eliminates the need for infrared (IR) camera arrays, reflective markers, or tracking hats.

To ensure the experience feels fluid and responsive, the system utilizes specialized digital smoothing. This provides stable, high-precision tracking that feels natural—without the lag or jitter typically associated with webcam-based setups.

What is the LPVISION-HeadPose Tracker?

At its core, the LPVISION-HeadPose Tracker is an intelligent software engine that understands how a user moves their head in real-time. Instead of relying on physical markers or heavy external sensors, it uses advanced AI to interpret natural facial structures and head movements through a standard RGB camera.

Moreover, the tracker is a core component of the LPVIZ ecosystem. In this capacity, it serves as a plug-and-play replacement for complex IR-based tracking setups, allowing users to access the full capability of LPVIZ with a significantly reduced hardware footprint.

Key Advantages Over Traditional IR Tracking

lpvision-hardware comparison

Historically, traditional high-quality motion capture has required bulky infrared camera arrays, room-scale calibration, and intrusive reflective markers or tracking hats. However, the LPVISION-HeadPose Tracker changes the equation across four key dimensions:

  • Markerless Operation via Facial Landmark Detection

First, the system utilizes AI-powered facial landmark detection and Computer Vision techniques to track head orientation and position continuously. Because of this, there is no rigid bodies, no reflective markers, and no physical contact with tracking gear required. Ultimately, this creates a non-intrusive experience.

  • Single RGB Camera Replaces Multi-Camera IR Arrays

Additionally, we have replaced expensive, multi-camera IR setups with a single, compact RGB camera — such as the Stereolabs ZED camera line. Therefore, this dramatically reduces hardware cost and physical setup complexity, while maintaining the precision required for industrial and spatial computing applications.

  • Zero-Contact Setup and Instant Deployment

Furthermore, you can go from zero to tracking instantly. In fact, there is no room-scale calibration, no physical mounting of tracking gear on the user, and no warm-up procedure needed. Consequently, the natural user experience means deployment in the field is as simple as launching the software.

  • Optimal for Vehicle-in-the-Loop and Cockpit Environments

Finally, unlike stationary IR camera arrays, this tracker is engineered for dynamic environments. Particularly, it is optimal for vehicle-in-the-loop (ViL) testing and flight cockpit applications — scenarios where traditional IR-based systems are difficult to setup due to limited space and dynamic lighting conditions.

System Performance at a Glance

 

The table below summarizes a comparison between LPVISION-HeadPose Tracker and Traditional Marker-Based Systems: 

Feature

LPVISION-HeadPose Tracker

Marker-Based  Tracking Systems

Setup effort

Instant – No markers needed

High – Requires reflective markers & calibration

User experience

Natural – No physical contact with gear

Intrusive – Users must wear tracking gear

Accuracy

High – Sub-cm error

Gold standard – Sub-millimeter error

Cost

Affordable – Standard RGB camera + software

Enterprise -Specialized IR camera + software

See It in Action

For a practical demonstration, the video below demonstrates real-time markerless head tracking using the LPVISION-HeadPose Tracker with a standard RGB camera in a live environment.

Designed for AR Glass-Style HMDs

Naturally, to maintain the highest level of accuracy, the LPVISION-HeadPose Tracker requires a clear line of sight to the user’s facial features. Therefore, this makes it the ideal companion for AR glass-style HMDs — sleek, open-face augmented reality devices where user comfort and natural interaction are the priority. For tracking standard VR headsets, please refer to our LPVR product line.

Integration with the LPVIZ Ecosystem

In addition, the LPVISION-HeadPose Tracker is a cornerstone of the LPVIZ platform. It is designed to act as a drop-in replacement for complex IR-based tracking setups, thereby enabling your team to enjoy the full power of LPVIZ with a significantly reduced hardware footprint.

Ultimately, by reducing the time and cost of industrial prototyping, we empower engineering and research teams to focus on innovation rather than calibration. Specifically, the tracker integrates directly into our existing LPVIZ workflow with no additional configuration required.

Ready to Transition to Markerless Tracking?

In conclusion, if you are building advanced AR or spatial computing systems and need precise, low-latency tracking that works in the real world, the LPVISION-HeadPose Tracker is ready for deployment. Contact our team today to schedule a live technical demo or to discuss how the LPVISION and LPVIZ systems can be integrated into your existing workflow.

Revolutionizing AD/ADAS Testing: VR-Enhanced Vehicle-in-the-Loop

The automotive industry is in a race to develop smarter, safer, and more efficient vehicles. To meet these demands, engineers rely on sophisticated development processes. At LP-RESEARCH, we’re committed to creating tools that shape the future of mobility. That’s why we’re excited to announce a groundbreaking collaboration with IPG Automotive.

By integrating our advanced hardware and software with IPG Automotive’s CarMaker, we’ve created an immersive Virtual Reality (VR) experience for Vehicle-in-the-Loop (ViL) testing. This powerful solution allows engineers to test Autonomous Driving and Advanced Driver-Assistance Systems (AD/ADAS) on a proving ground with unprecedented realism and efficiency.

What is Vehicle-in-the-Loop (ViL)?

Vehicle-in-the-Loop is a powerful testing method that blends real-world driving with virtual simulation. An AD/ADAS-equipped vehicle drives on a physical test track while interacting with a dynamic virtual environment in real time.

This approach lets engineers observe the vehicle’s response to countless simulated scenarios under controlled, repeatable conditions. The vehicle’s real-world dynamics are continuously fed back into the simulation, ensuring the virtual world perfectly mirrors the physical state of the car. The test vehicle is outfitted with a seamless integration of hardware and software to support this constant flow of data.

The Technology Stack: A Powerful Combination

Our collaboration combines best-in-class hardware and software from both LP-RESEARCH and IPG Automotive to deliver a complete ViL solution.

LP-RESEARCH Stack

LPVR-POS: Our hardware system for acquiring physical data from the vehicle via ELM327 or OBDLink, completed by Global Navigation Satellite System (GNSS) antennas, and advanced Inertial Measurement Units (IMUs). LPPOS includes FusionHub, our sensor-fusion software for high-precision, real-time vehicle state estimation.

LPVR-DUO: Specialized sensor-fusion software that calculates the Head-Mounted Display (HMD) pose relative to the moving vehicle.

ART SMARTTRACK3: An advanced Infrared (IR) camera tracking system from our partner, Advance Realtime Tracking (ART), for precise head tracking.

IPG Automotive Stack

CarMaker Office: The simulation environment, enabled with the ViL add-on.

Movie NX: The visualization tool, enhanced with the VR add-on to create an immersive experience.

How It All Works Together

The key to this integration is a custom plugin that connects out FusionHub software with CarMaker. This plugin translates the real vehicle’s precise position and orientation (it’s “pose“) into the virtual environment.

The system workflow is a seamless loop of data capture, processing, and visualization:

Data Acquisition: LPPOS gathers vehicle data (OBD), GNSS, and IMU measurements and sends it to FusionHub. The SMARTRACK system monitors the HMD’s position, while IMUs on the headset and vehicle platform send orientation data to LPVR-DUO.

Sensor Fusion: FusionHub processes its inputs to calculate the vehicle’s exact pose in the real world. LPVR-DUO calculates the HMD’s pose relative to the moving vehicle’s interior.

Real-Time Communication: FusionHub streams the vehicle’s pose to a dedicated TCP server, which feeds the data directly into the CarMaker simulation via our custom plugin. LPVR-DUO communicates the headset’s pose to Movie NX using OpenVR, allowing the driver or engineer to naturally look around the virtual scene from inside the real car.

The entire LP-RESEARCH software stack and CarMaker Office run concurrently on a single computer inside the test vehicle, creating a compact and powerful setup.

See It in Action

We configured a scene in the CarMaker Scenario Editor that meticulously replicates our test track near the LP-RESEARCH Tokyo Office. The video on the top of this post demonstrates the fully integrated system, showcasing how the vehicle’s real-world position perfectly matches its virtual counterpart. Notice how the VR perspective shifts smoothly as the copilot moves their head inside the vehicle.

This setup vividly illustrates how VR technology makes ViL testing more immersive, effective, and even fun.

Advance Your ViL Testing Today

Are you ready to integrate cutting-edge virtual reality into your Vehicle-in-the-Loop testing and help shape the future of mobility?

Fine-tuning the HMD view and virtual vehicle reference frame is crucial for an accurate simulation and depends on the specific test vehicle and scenario. Our team has the expertise to configure these parameters for you or provide expert guidance to ensure a perfect setup.

Contact us today to learn how our tailored solutions and expert support can elevate your AD/ADAS development process.