LPVR-AIR Demo at SIGGRAPH 2023

28/08/2023
, , , , , Blog, Event

Ready – Steady – Go!

We had an exciting week at the Siggraph 2023 computer graphics conference in Los Angeles showcasing LPVR-AIR! Together with optical tracking systems maker Optitrack and content creator Meptik we developed a real-time racing experience using trikes. On these trikes exhibition attendees were able to race in a small concourse we set up at the exhibition site. Each trike rider we equipped with a Meta Quest virtual reality headset showing a Mario Kart-like track using our LPVR-AIR tracking and real-time streaming solution.

The twist: We attached an Optitrack active marker target to each HMD so we were able to have multiple players move in a globally referenced tracking volume. With LPVR-AIR we don’t render the 3D content on the HMDs themselves, but wirelessly transmit it from specialized rendering PCs. This allows us to display high polygon-count content on the HMDs without exceeding the limited computing power of the standalone headsets.

We are still working on a dedicated product page for LPVR-AIR, but already created a solid amount of documentation how to use the solution. Have a look at it here. The document also gives an insight into the specialities and limitations of the system.

Use-Cases Outside Gaming

LPVR series solutions are the industry standard for large-area VR and AR tracking. They enable consumer and professional headsets to work in conjunction with professional-grade tracking and motion capture systems, leveraging their power both in terms of precision and coverage of large areas. While in this demonstration, the trikes were tracked from the outside, we also enable headset tracking completely contained inside moving platforms such as simulators or cars. We are constantly developing and updating it according to our customers’ needs.

Thanks to everybody who stopped by and pedaled their way around the course. Great riding, well done!

New Features in LPVR Version 4.8

02/08/2023
, , , , Blog, Product

Introduction

Our LPVR series is the primary solution on the market for users who want to expand the scope of their virtual reality or mixed reality headsets by using external tracking systems such as ART, OptiTrack or Vicon. Use cases are varied and range from entertainment (location based VR) and engineering use cases (ergonomic studies in AR) to helicopters and virtual cars which are actually driving on Japan’s public roads. At LP-Research, we have continuously developed the LPVR series of solutions over the past years. We have expanded its scope, added support for new headsets, and included new functions.

The image below shows an LPVR installation based on design content created by automotive prototyping company Phiaro Inc. in Tokyo, Japan.

The latest release is version 4.8.0, which we released in June of 2023.  As usual, it comes in two flavors:

  • LPVR-CAD which supports stationary use-cases, and
  • LPVR-DUO which is our variant for moving platforms, be they cars or simulators.

We support all the major tethered headsets (SteamVR headsets, Pimax, Varjo).  We also support Meta Quest series headsets and the Vive Focus 3 with our LPVR-Air series of products. If you have a current support contract, you are eligible for an update.

A Brief Overview of LPVR-CAD and LPVR-DUO

It’s maybe best to summarize some of the capabilites that our products add to the various commercial headsets.  For more details, feel free to visit the product pages for LPVR-CAD and LPVR-DUO, respectively:

  • Cover arbitrary large areas and have VR scenes taking place in them
  • Have an arbitrary number of users interact in such a space
  • Do VR/AR inside a car or any other moving platform
  • Track your user to sub-millimeter precision together with any number of props with no perceivable latency
  • Use SteamVR controllers without the Lighthouses

We can do this because our proprietary sensor fusion algorithms allow us to combine the measurements of high-precision motion tracking camera systems with the measurements of the headset’s Inertial Measurement Unit (IMU). For the case of a moving platform, we can additionally incorporate data from an IMU installed on the platform to provide for a responsive, accurate performance also in those circumstances.

New Features

For a short overview of the changes in each version, please refer to our Release Notes. Here we will give some highlights and dig into some details. LPVR 4.8.0 is the result of continuous development in the half year or so since our previous releases.

New GUI Organization and Visual LPVR-DUO Configuration Interface

The most obvious change to users will be the reorganized GUI which streamlines the setup, completely doing away with the need to enter any JSON codes, while coming on a more cleanly organized surface. Especially for our LPVR-DUO users this means a vast simplification of the system.  We have maintained the old configuration interface as an option to guarantee compatibility with existing workflows, but we don’t think that users will have to resort to it. Please let us know if your experience is different. If your headset tracking body is already calibrated, you should now be able to setup LPVR-DUO with some five mouse clicks.

When you load up the configuration, it will look something like this. Note that you no longer are led to a JSON editor where you manually have to enter the configuration. Instead you are greeted by a friendly, informative GUI.

At the bottom of the page, you will see links to the Documentation, a Calibration screen, and an Expert Mode, basically the old JSON editor. The Calibration screen is used for the setup of the Platform IMU and simplifies it down to a few mouse clicks in the usual case. No more looking for some quaternion values in log files! Please check out the corresponding documentation.

Varjo Headset Eye Point Adjustments

Together with Varjo and with cooperation of several of our customers we were able to identify and correct some imprecisions in the handling of the headset’s position. These would show up as small coordinate mismatches between the optical tracking coordinates and the coordinates reported to VRED or Unity etc. Additionally, this would lead to some unnatural motion of AR overlays, especially when turning the head.

Optimal performance requires updating both Varjo Base to at least version 3.10 and LPVR to at least version 4.8.0.  Updating Varjo Base fixes the underlying issue, updating LPVR corrects the interfacing.  If you cannot update Varjo Base, you can still update LPVR-CAD-Varjo to version 4.8.0 and enable a workaround.  To do so, please open the Varjo Base configuration GUI on the System tab and then add patchPositionBug=true in the field labeled Additional Settings followed by clicking the “Submit” button. Note while this works around the issue in Varjo Base before version 3.10, it is not recommended to use this option with the updated versions of Varjo Base.

Varjo Configuration Refinements

Different environments call for different setups.  Some of our users use administrator accounts, others have multiple users but want them to use the same configuration.  We have updated the way we organize on-disc storage of the configuration to address these possibilities.  In particular you can now establish a system-wide configuration default, and you can override it per-user.  In the case of LPVR-CAD, additionally, the configuration is entered inside Varjo Base by default, but to allow users greater flexibility, it has always been possible to use our web interface or files on disk to perform the configuration.  While these are not the preferred choice, it was previously not possible to see from Varjo Base whether the on-disk configuration is in use.  We have added a prominent status information that points to the configuration, as in the screen shot below.  In the case of LPVR-DUO the configuration is always loaded from disk as the added flexibility of our configuration page is required, but in LPVR-CAD the user will have to opt in. We describe the process briefly below.

The user can prepare a global, system-wide default configuration in %ProgramData%/Varjo/VarjoTracking/Plugins/LP-Research/LPVR-CAD-Varjo/configuration/settings.json. Changes on the configuration page will not change this configuration, but will instead be written to the per-user configuration %LocalAppData%/LP-Research/LPVR-CAD-Varjo/settings.json. If either file is present, the configuration inside Varjo Base will be ignored and the user can use their web browser to configure LPVR-CAD. In this way, an administrator can prepare a configuration that works with the setup, and any user can customize it to their needs. For LPVR-DUO, there is no configuration interface inside Varjo Base, instead the user will always point their web browser to http://localhost:7119. Here, a system-wide default configuration can be placed in %ProgramData%/Varjo/VarjoTracking/Plugins/LP-Research/LPVR-DUO-Varjo/configuration/settings.json, and a per-user override can sit in %LocalAppData%/LP-Research/LPVR-DUO-Varjo/settings.json. The web interface will always update this per-user file.

LPVR-DUO Demonstration

In order to familiarize you with the neighborhood of our office and, more importantly, to show what can be done with LPVR-DUO, here is an in-car mixed reality demonstration. The video screens on the glove box may look almost real but they are an overlay imposed on the see-through camera image of a Varjo XR-3 using an out-of-the-box LPVR-DUO set. Notice how the screens firmly remain in place during turns of the user’s head as well as turns of the car itself, even when diving into some of the steeper roads of the Motoazabu area in central Tokyo.

Exploring Affective Computing Concepts

02/11/2021
Blog, Projects

Introduction

Emotional computing isn’t a new field of research. For decades computer scientists have worked on modelling, measuring and actuating human emotion. The goal of doing this accurately and predictively has so far been elusive.

pngwing.com

In the past years we have worked with the company Qualcomm to create intellectual property related to this topic, in the context of health care and the automotive space. Even though this project is pretty off-topic from our ususal focus areas it is an interesting sidetrack that I think is worth posting about.

Affective Computing Concepts

As part of the program we have worked on various ideas ranging from relatively simple sensory devices to complete affective control systems to control the emotional state of a user. Two examples of these approaches to emotional computing are shown below.

The Skin Color Sensor measures the color of the facial complexion of a user, with the goal of estimating aspects of the emotional state of the person from this data. The sensor is to have the shape of a small, unobtrusive patch to be attached to a spot on the forehead of the user.

Another affective computing concept we have worked on is the Affectactic Engine. A little device that measures the emotional state of a user via an electromyography sensor and accelerometer. Simply speaking we are imagining that high muscle tension and certain motion patterns correspond to a stressed emotional state of the user or represent a “twitch” a user might have.

The user is to be reminded of entering this “stressed” emotional state by vibrations emitted from the device. The device is to be attached to the body of the user by a wrist band, with the goal of reminding the user of certain subconscious stress states.

Patents

In the course of this collaboration we created several groundbreaking patents in the area of affective computing:

コネクシオ 設備監視IoTシステムについて

27/08/2021
Blog, Use Cases

概要

今回はコネクシオ様によって開発された工場等における設備監視IoTシステムについてご紹介します。本システムは工場設備の異常振動等を検知・モニタリングすることで故障予測や設備状況の把握することを目的としています。
振動を検知するために弊社の製品LPMS-IG1が使用されています。

 

 

使用事例：排水処理施設設備の監視

K社では廃水処理プラント施設を運用しています。本施設では有機性の浮遊物や油分が大量に含まれるビルピット汚泥など処理しづらい廃水を「メタン発酵処理」「活性汚泥処理」方法等を組み合わせたシステムにより物理的、生物化学的に分解処理を行っています。

問題

排水処理プラントは365日24時間稼働し続ける必要あります。
また、微生物の働きによって分解を行うため、設備を停止することはできません。そのような状況下において、深夜・早朝休祝日関わらず発生する突発的な故障や緊急対応の業務負荷が大きな課題となっています。

解決

コネクシオ様と協力し、IoT設備監視システムを実験的に導入。
連続データの見える化により、人的監視では困難な解析や包括的な状態把握が可能となりました。遠隔からポンプの動作状況が確認でき稼働率、時系列での状態変化の把握が可能になった他、現場に行かなくても状態を確認できるため、保守点検・巡回業務の稼働が軽減しました。

 

実際の機器の写真(※1)

 

システム構成図(※1)

 

 

 

本システムにおけるLPMS-IG1

センサーフュージョン機能を搭載した MEMS9 軸 IMU「LPMS-IG1」は弊社製品の中でも最高精度のセンサとなっています。低ノイズ/低ドリフト (4℃/h) の加速度センサに加え、2 種類のジャイロセンサを搭載することにより、幅広い範囲での計測（ 400 2000 °/s）が可能です。詳細は製品紹介ページ（日本語）をご確認下さい。

詳細はこちら

耐衝撃性＋防水

LPMS-IG1は10,000Gの衝撃を耐える耐久性、IP67の防水機能を有しています。排水プラントのような水に関わる用途や、強い衝撃・振動が発生するケースでも問題なく使用することが可能です。

 

高いカスタマイズ性

今回のようなケースでは振動検知の精度が求められます。より幅広い数値範囲での振動を検知するため、ローパスフィルタのしきい値設定を変更。故障の予兆となりうる比較的小さな振動も検知できるようになりました。LPMS-IG1は設定ファイルを変更することで簡単にセンサのパラメータを変更することができます。

 

顧客の問題を解決

LP-Researchのサービスはセンサを販売して終わるわけではありません。
開発を進めていく中でセンサにまつわる様々な問題・不明点等が発生します。弊社ではお客様の問題を解決すべく全力でサポートしております。今回のコネクシオ様の開発においては開発に関する問い合わせ対応はもちろん、システムに必要となったパーツも弊社で開発しました。

開発したパーツ１：複数センサを接続できるCANケーブル

 

開発したパーツ２：センサをパイプに設置するためのアタッチメント

 

※1 出典：廃棄物処理施設でのモーター故障検知（K社様）

How to Connect an LP-Research IMU to ROS (Update)

04/01/2021
, , Blog, Product

Introduction

This article describes how to connect an LP-RESEARCH inertial measurement unit (IMU) using a Robot Operating System (ROS) node. We are happy to announce that our IMU ROS sensor driver has been accepted into the official ROS package repository. The Robot Operating System, or ROS in short, is an open-source de-facto standard for robotics sensing and control.

With the package openzen_sensor now provided as part of the ROS distribution Melodic Morenia it just became a whole lot easier to use our sensors in robotic applications.

Note: This article covers our node for ROS 1. Please see further information regarding our ROS 2 node at the end of this article. This post is a follow-up to our previous ROS driver release.

Published ROS Topics

These are the ROS topics which are published by the OpenZen ROS driver:

Message

Type

Description

/imu/data

sensor_msgs/Imu

Inertial data from the IMU. Includes calibrated acceleration, calibrated angular rates and orientation. The orientation is always unit quaternion.

/imu/mag

sensor_msgs/MagneticField

Magnetometer reading from the sensor.

/imu/nav

sensor_msgs/NavSatFix

Global position from a satellite navigation system. Only available if the IMU includes a GNSS chip.

/imu/is_autocalibration_active

std_msgs/Bool

Latched topic indicating if the gyro autocalibration feature is active.

Installation of the LPMS ROS Driver

All that’s needed is to install the package openzen_sensor via your Linux distribution’s package manager. In Ubuntu, with the ROS Melodic Morenia distribution installed, use the following command:

apt install ros-melodic-openzen-sensor

Once the IMU ROS driver package is installed, we use the following command to start the OpenZen node:

rosrun openzen_sensor openzen_sensor_node

This will automatically connect to the first available IMU and start streaming its accelerometer, gyroscope and magnetometer data to ROS. If your sensor is equipped with a GPS unit, global positioning information will also be transferred to ROS.

Once a sensor has been connected via the motion sensor driver, the data from the sensor is exported via ROS topics which can be consumed by other ROS components such as a navigation and path planning system.

Outputting IMU sensor values on the command line can now be easily done with:

rostopic echo /imu/data

and the data can be plotted with:

rosrun rqt_plot rqt_plot /imu/data/linear_acceleration

More information on the usage of the OpenZen IMU ROS driver can be found in the repository of the driver.

The image above shows an angular velocity output graph in the ROS MatPlot application from an LPMS-IG1 sensor.

ROS 2 Release

We have recently released a ROS 2 version of our OpenZEN ROS node. The node is not part of an official ROS2 release yet, but it works well on the latest release Foxy. For surther information and source code see the OpenZenROS2 repository.

1 2 3 4 5 … 15