Do you need help? Please contact our support team from 9 to 17 CET via

PressClub Portugal · Article.

BMW Group and Meta’s Reality Labs present joint research for interlinking extended reality devices with the digital vehicle ecosystem.

+++ Pioneering research for potential industry standardization +++ Extended reality (XR) consumer devices could become increasingly important for customers in the future +++ Research prototype overcomes key technical challenges +++

Infotainment, Conceitos de Controlo
Sistemas de Motorização Alternativos, Mobilidade do Futuro

Press Contact.

Joao Trincheiras
BMW Group

Tel: +351-21-487-3080

send an e-mail

Munich / Mountain View . The BMW Group Technology Office USA in Mountain View presents a research update with Meta’s Reality Labs Research, demonstrating, for the first time, the ability to accurately display stable virtual reality (VR) and mixed reality (MR) content to passengers in a fast-moving car, even when making turns, going over speed bumps, and accelerating. By integrating Meta Quest’s tracking system with BMW’s sensor data, the researchers enabled a series of stable “car-locked” gaming, entertainment, productivity, and meditation experiences.

Claus Dorrer, Head of BMW Group Technology Office USA in Mountain View: “It is too early to tell exactly how or when this technology will make it into customers’ hands, but we envision a number of potential use cases for XR devices in vehicles — from assisting the driver in locating their car in a crowded parking lot to alerting them to hazards on the road and surfacing important information about the vehicle’s condition. The implications of future AR glasses and VR devices — for passengers as well as drivers — are promising. The research partnership with Meta will allow us to discover what immersive, in-vehicle XR experiences could look like in the future and spearhead the seamless integration of such devices into cars.”

Proof-of-concept prototype overcomes key technical challenges.
VR headsets use cameras and motion sensors to understand their exact position, so when the user moves their head or looks around, the virtual content remains stable. However, in fast-moving vehicles, the headset’s sensors get confused by conflicting signals from the device’s movement within the car and the vehicle’s movement in the world. Simply put, the headset’s motion sensors think it is moving, but its cameras think it is standing still when looking at the car’s interior.

That mismatch means today’s VR headsets can’t display stable virtual content when travelling inside a fast-moving vehicle, particularly if the vehicle accelerates, turns fast, or encounters bumps on the road.

To solve this challenge, BMW and Meta researchers collaborated to incorporate information from a BMW car’s sensor array into the tracking system of a Meta Quest Pro. That allowed the tracking system to simultaneously calculate the device’s location relative to the car and the moving car’s location relative to the world, enabling it to accurately anchor virtual objects to a moving car and demo compelling VR and MR passenger experiences.

Richard Newcombe, Vice President of Research Science, Reality Labs Research: “Our research prototype shows that we can enable entertaining and comfortable passenger experiences that are anchored to the car itself, including VR and MR gaming, entertainment, productivity, and even meditation capabilities. The technology has the potential to transform how we can safely interact with our environment while traveling, and as we progress into reliable world-locked content on the road to AR glasses, we’re hopeful it will be possible for passengers to see things like markers for landmarks, restaurants, places of interest, and more.”

Combining the movement data generated by the vehicle with XR headset sensor capabilities allowed us to subtract the vehicle motion and more accurately track the headset’s movement in a vehicle. That in turn enabled us to render content which is synced to the movement of the vehicle in real-time, both inside and outside of the vehicle at high speeds, regardless of whether the vehicle is on a flat, straight street or a bumpy road with sharp turns.

This proof-of-concept prototype demonstrates an in-car full range of a person’s movements (known as six degrees of freedom) and an XR experience that simultaneously calculates a headset’s location relative to the car and the moving car’s location relative to the world by integrating with the car’s sensors.

The BMW Group has been successful in pioneering digital technologies like vehicle access based on the Digital Key and in driving industry standardization for such innovations. The company’s global network of technology offices has been crucial in kickstarting such innovation projects. The BMW Group is open to including additional partners in order to achieve the goal of creating an industry standard for connecting XR consumer electronics devices to vehicles. For potential series development, the BMW Group is committed to the privacy-by-design principle. This means, in all digital feature development, the BMW Group takes steps to ensure that customer data is protected and securely processed in accordance with data privacy requirements by means of established processes — across all the markets in which the company operates.

Link to Meta Reality Labs information on the project.

Article Offline Attachments.

Article Media Material.

My.PressClub Login

BMW Group Streaming


Beijing. 24/25 April 2024.

Here you can see the Webcasts of the BMW Group Night and Press Conference at the Auto China 2024; with the World Premieres of the new MINI Aceman and the new BMW i4.

Open Streaming Page

CO2 emission information.

Os valores do consumo de combustível, das emissões de CO2 e de consumo de energia aqui apresentados foram determinados de acordo com o Regulamento (CE) 715/2007 na versão aplicável no momento da homologação. Os valores referem-se à versão base de um veículo na Alemanha e na gama apresentada é considerado o equipamento opcional e os diferentes tamanho de jantes e pneus para a versão selecionada.

Os valores dos veículos já se baseiam no novo Regulamento WLTP e são traduzidos para valores equivalentes aos do NEDC de modo a garantir a comparação entre os veículos. [Relativamente a estes veículos, no que se refere a impostos relacionados com o veículo ou outras taxas baseadas (no mínimo, entre outros) nas emissões de CO2, os valores de CO2 podem ser diferentes dos valores aqui apresentados.]

As especificações da eficiência de CO2 são determinadas de acordo com a Diretiva 1999/94/CE e o regulamento europeu na sua versão atual aplicável. Os valores apresentados são baseados no consumo de combustível, valores de CO2 e de consumo de energia de acordo com o ciclo NEDC para a classificação.

Para mais informações sobre os dados oficiais do consumo de combustível e as emissões específicas de CO2 dos automóveis de passageiros novos, consulte as diretrizes sobre o consumo de combustível, emissões de CO2 e consumo de energia elétrica de automóveis de passageiros novos, disponíveis gratuitamente em todos os Concessionários BMW.

Add your filter tags.

Informação à Imprensa
Dossier de imprensa
Fact & Figures
Submit filter
Clear all