Do you need help? Please contact our support team from 9 to 17 CET via support.pressclub@bmwgroup.com.

PressClub Global · Article.

BMW Group and Meta’s Reality Labs present joint research for interlinking extended reality devices with the digital vehicle ecosystem.

+++ Pioneering research for potential industry standardization +++ Extended reality (XR) consumer devices could become increasingly important for customers in the future +++ Research prototype overcomes key technical challenges +++

Technology
·
Infotainment, Control Concepts
·
Mobility of the future
 

Munich / Mountain View . The BMW Group Technology Office USA in Mountain View presents a research update with Meta’s Reality Labs Research, demonstrating, for the first time, the ability to accurately display stable virtual reality (VR) and mixed reality (MR) content to passengers in a fast-moving car, even when making turns, going over speed bumps, and accelerating. By integrating Meta Quest’s tracking system with BMW’s sensor data, the researchers enabled a series of stable “car-locked” gaming, entertainment, productivity, and meditation experiences.

Claus Dorrer, Head of BMW Group Technology Office USA in Mountain View: “It is too early to tell exactly how or when this technology will make it into customers’ hands, but we envision a number of potential use cases for XR devices in vehicles — from assisting the driver in locating their car in a crowded parking lot to alerting them to hazards on the road and surfacing important information about the vehicle’s condition. The implications of future AR glasses and VR devices — for passengers as well as drivers — are promising. The research partnership with Meta will allow us to discover what immersive, in-vehicle XR experiences could look like in the future and spearhead the seamless integration of such devices into cars.”

Proof-of-concept prototype overcomes key technical challenges.
VR headsets use cameras and motion sensors to understand their exact position, so when the user moves their head or looks around, the virtual content remains stable. However, in fast-moving vehicles, the headset’s sensors get confused by conflicting signals from the device’s movement within the car and the vehicle’s movement in the world. Simply put, the headset’s motion sensors think it is moving, but its cameras think it is standing still when looking at the car’s interior.

That mismatch means today’s VR headsets can’t display stable virtual content when travelling inside a fast-moving vehicle, particularly if the vehicle accelerates, turns fast, or encounters bumps on the road.

To solve this challenge, BMW and Meta researchers collaborated to incorporate information from a BMW car’s sensor array into the tracking system of a Meta Quest Pro. That allowed the tracking system to simultaneously calculate the device’s location relative to the car and the moving car’s location relative to the world, enabling it to accurately anchor virtual objects to a moving car and demo compelling VR and MR passenger experiences.

Richard Newcombe, Vice President of Research Science, Reality Labs Research: “Our research prototype shows that we can enable entertaining and comfortable passenger experiences that are anchored to the car itself, including VR and MR gaming, entertainment, productivity, and even meditation capabilities. The technology has the potential to transform how we can safely interact with our environment while traveling, and as we progress into reliable world-locked content on the road to AR glasses, we’re hopeful it will be possible for passengers to see things like markers for landmarks, restaurants, places of interest, and more.”

Combining the movement data generated by the vehicle with XR headset sensor capabilities allowed us to subtract the vehicle motion and more accurately track the headset’s movement in a vehicle. That in turn enabled us to render content which is synced to the movement of the vehicle in real-time, both inside and outside of the vehicle at high speeds, regardless of whether the vehicle is on a flat, straight street or a bumpy road with sharp turns.

This proof-of-concept prototype demonstrates an in-car full range of a person’s movements (known as six degrees of freedom) and an XR experience that simultaneously calculates a headset’s location relative to the car and the moving car’s location relative to the world by integrating with the car’s sensors.

The BMW Group has been successful in pioneering digital technologies like vehicle access based on the Digital Key and in driving industry standardization for such innovations. The company’s global network of technology offices has been crucial in kickstarting such innovation projects. The BMW Group is open to including additional partners in order to achieve the goal of creating an industry standard for connecting XR consumer electronics devices to vehicles. For potential series development, the BMW Group is committed to the privacy-by-design principle. This means, in all digital feature development, the BMW Group takes steps to ensure that customer data is protected and securely processed in accordance with data privacy requirements by means of established processes — across all the markets in which the company operates.

Link to Meta Reality Labs information on the project.

Article Offline Attachments.

Article Media Material.

My.PressClub Login
 

BMW Group Streaming

ANNUAL CONFERENCE 2024.

Munich. Here you can see the on demand webcast of the BMW Group Annual Conference 2024, with the world premiere of the next BMW vision vehicle on the "Road to Neue Klasse".

Open Streaming Page

CO2 emission information.

Fuel consumption, CO2 emission figures and power consumption and range were measured using the methods required according to Regulation VO (EC) 2007/715 as amended. They refer to vehicles on the German automotive market. For ranges, the NEDC figures take into account differences in the selected wheel and tyre size, while the WLTP figures take into account the effects of any optional equipment.

All figures have already been calculated based on the new WLTP test cycle. NEDC figures listed have been adjusted to the NEDC measurement method where applicable. WLTP values are used as a basis for the assessment of taxes and other vehicle-related duties which are (also) based on CO2 emissions and, where applicable, for the purposes of vehicle-specific subsidies. Further information on the WLTP and NEDC measurement procedures is also available at www.bmw.de/wltp.

For further details of the official fuel consumption figures and official specific CO2 emissions of new cars, please refer to the “Manual on the fuel consumption, CO2 emissions and power consumption of new cars”, available at sales outlets free of charge, from Deutsche Automobil Treuhand GmbH (DAT), Hellmuth-Hirth-Str. 1, 73760 Ostfildern-Scharnhausen and at https://www.dat.de/co2/.

Latest Facts & Figures.

Specifications of the BMW XM 50e, valid from 02/2024.
Wed Mar 06 13:48:00 CET 2024
G09
BMW M Automobiles
Specifications of the BMW i5 xDrive40, valid from 03/2024.
Thu Feb 29 15:53:00 CET 2024
G60
BMW i
Specifications of the BMW iX2 eDrive20, valid from 03/2024.
Sat Feb 17 09:00:00 CET 2024
U10
BMW i
Show all

Add your filter tags.

Press Release
Press Kit
Speech
Fact & Figures
Spanish
  • Language of file attachment
  • Arabic
  • Chinese
  • Chinese (simplified)
  • English
  • English
  • French
  • German
  • Hungarian
  • Italian
  • Japanese
  • Korean
  • Other Language
  • Russian
  • Spanish
Updates
Top-Topic
Submit filter
Clear all
 
Sun
Mon
Tue
Wed
Thu
Fri
Sat
 
Login