Automated driving at the BMW Group.

1.Introduction.

Personal mobility and its industrial application are on the brink of a technological quantum leap. The car and the technologies that make it tick are set to change more in the next 10 years than they have in the past 30 combined. The BMW Group sees itself as being well equipped to meet this challenge.

Over the course of recent years, the BMW Group has already demonstrated the basic essentials needed for autonomous driving at various events. As long ago as 2006, a BMW 3 Series was already lapping the circuit at Hockenheim by itself, while automated prototypes from the BMW Group started undergoing road tests on the A9 motorway between Munich and Nuremberg in 2011. Since 2014, more advanced prototypes from the BMW Group have been equipped with 360° environmental sensing technology, giving them the ability to avoid accidents. 2014 was also the year that an automated prototype from the BMW Group drifted its way around the Las Vegas Speedway for the first time, proving that even driving at the limits of performance is within the realms of possibility. Automated vehicles are now even capable of parking themselves when prompted to using simple gestures (automated valet parking).

The BMW Vision Next 100 is a visionary vehicle that incorporates all of these functions, at the same time as helping owners to manage their daily routine.
The BMW Group has in the meantime gone one step further: as a result of its acquisition of a stake in HERE in 2014 and it collaborations with Intel, Mobileye (since 2016) and other partners, the BMW Group has given the go-ahead for developing the BMW iNext for series production.

The BMW Group and its partners are working together on standards, platforms and a backend for the future of automated driving and would be delighted to welcome new partners into their alliance at any time.

2.(Highly) Automated driving at the BMW Group.
History and current status.

2006: The first self-driving Track Trainer (BMW 3 Series) laps Hockenheim while following the racing line.
Since 2011: Highly automated test cars on the A9 motorway in Germany.
2014 CES: Drift Assistant demonstrates perfect vehicle control even when driving at the limits of performance.
2015 CES: 360° collision prevention and Remote Valet Parking Assistant presented in the BMW i3.
2016 CES: Automated Gesture Control Parking in the BMW i3.

3.Driver assistance systems (level 2)and automated/autonomousdriving (levels 3 to 5).

SUMMARY.

Level 2 (today):

Driver assistance systems as preliminary stage of automated driving. The driver is responsible for the task of driving at all times (hands-on detection).

Level 3 (starting from 2021 with BMW iNext):

Once level 3 is reached, it will be possible for driver and vehicle to share the responsibility for controlling the vehicle for the first time. During highly automated driving in traffic that is moving in the same direction and is segregated from oncoming traffic, the driver will be able to perform secondary in-vehicle activities for longer periods of time or relax (eyes off). They must still be in a position to take over the task of driving again within a reasonable amount of time (a few seconds) when prompted to by the system.

Level 4 (starting from 2021 with technical provisos, BMW iNext):

Fully automated driving in urban traffic and – in a version with extended functionality – in traffic that is moving in the same direction and is segregated from oncoming traffic. The driver can sleep during long-distance journeys if necessary. The key difference compared to level 3: the time span for taking over control again is far longer (mind off).

Level 5 (developments in parallel to levels 3 and 4 probably possible post-2020 in the form of pilot projects):

Autonomous driving, steering wheel and pedals no longer absolutely necessary, passengers sit in the vehicle without any involvement in the task of driving; driving licence not required (driver off). Assuming the vehicle is fitted with pedals and a steering wheel, the driver may take over the task of driving if they wish but will never be obliged to do so.

THE AUTOMATION LEVELS IN DETAIL.

Level 2 (driver assistance systems today).

Of the five levels of vehicle automation defined by the automotive industry, production vehicles today are at level 2 (driver assistance systems). The BMW Group deliberately refers to the systems as driver assistance systems, such as Driving Assistant Plus in the new BMW 7 Series and BMW 5 Series. Assistance with the longitudinal and lateral guidance of the vehicle enables relaxed, safer driving, meaning that far less strain is placed on the driver. The current systems nevertheless require the driver to permanently focus their attention on the traffic, partly due to the capabilities of today’s technology as well as the legal situation at present. The driver is responsible for the task of driving at all times. Features such as hands-on detection are designed to ensure this responsibility is fulfilled.

Level 3 (highly automated driving).

Vehicles at this level will be capable of completely automated driving and therefore of taking over control in segregated traffic that is moving in the same direction, for example when driving on motorways or motorway-type roads. During this time, the driver will be able to perform other activities (“eyes off”), but in complex situations they must be in a position to take over the task of driving again within a reasonable amount of time (a few seconds) when prompted to by the system. One example here would be a new set of roadworks with a confusing layout and complex lane routing. In any case, the driver needs a driving licence and must be capable of driving a motor vehicle.

Level 4 (fully automated driving).

From a technological standpoint, level 4 represents the evolutionary progression from level 3. The driver will only have to take over the driving either in extremely complex situations or when extreme weather conditions are expected ahead. Although the driver must still possess a valid driving licence at this level and be physically fit to drive, driving in “mind off” mode is theoretically possible, which would allow the driver to sleep during the journey.

The main differences between level 3 and level 4 are the timeframe during which the driver must take over the task of driving and the ability to negotiate urban traffic in fully automated mode at level 4. At level 3, the driver must assume control again within a reasonable amount of time (a few seconds). If they fail to do so, the vehicle will automatically put itself in a low-risk situation, for example by braking to a controlled stop on the hard shoulder. There will be far more time for taking over control during automated driving at level 4.

The key difference between level 4 and level 5 is that although the driver may take over the driving if they wish at level 5, they will never be obliged to do so.

Level 5 (autonomous driving).

In contrast to levels 3 and 4, neither fitness to drive nor a driving licence is required for autonomous driving, dispensing with the need for a steering wheel and pedals. The vehicle takes over all driving functions. This is exceedingly complex, so the demands placed on technical solutions are extremely high. For this reason, self-driving vehicles will initially operate at relatively low speeds in urban traffic. To begin with, they will therefore be primarily deployed in city centres, where they will be used in delimited areas at first.

It is conceivable that the first self-driving vehicles will be brought out alongside highly automated vehicles in the period between 2020 and 2030. It is expected that the first pilot projects with autonomous vehicles will be launched in selected city centres during the course of this decade. Giving a definite date for when this could happen would be pure speculation, however. So, highly automated vehicles will appear on motorways initially, while autonomous vehicles will start off in city centres as part of pilot projects launched in parallel.

3.1. Technology and technical requirements of today’s prototypes. 

Highly automated driving (level 3) brings with it a whole series of technological requirements and is only made possible by the perfect interaction of each and every component. To this end, the individual sensors relay data, which is then amalgamated to produce a 360° environment model of the vehicle’s surroundings. Based on this, the driving strategy software computes the necessary driving manoeuvres.

- Laser scanners precisely measure the distances to other objects in thevehicle’s surrounding area and determine both their size and speed. In thisway, the vehicle generates an image of which areas are passable and clear ofobstacles.

- The cameras installed behind the windscreen determine the position of otherroad users and recognise whether the road user in question is a car, truck,motorcycle or pedestrian. The camera detects road markings too, enabling itto provide exact information on the vehicle’s position within its own lane.

- The radar sensors that are directed towards both the front and rear map thepositions of other road users. They detect the direction that objects areapproaching from and continuously calculate the distance to them as well astheir speed.

- The ultrasonic sensors detect other vehicles as well as obstacles in thevehicle’s immediate vicinity. There are sensors on each side of the vehicle aswell as at the front and rear for this purpose.

- The GPS (Global Positioning System) is used together with information fromthe on-board sensors to locate the vehicle’s position on a high-precision HDmap. This includes information on the number of traffic lanes and on accessroads or exit roads, for instance, as well as “landmarks” that have beenmeasured exactly. In this way, the vehicle’s position can be pinpointed to theexact lane. And by factoring in the data from the camera on the interior mirror,the vehicle can work out how close it is to the lane markings or the edge ofthe road.

The “data centre” for processing all the information that is received can currently be found in the boot of every prototype. This is where the driving strategy is computed on the basis of the information collected. The strategy specifies how the vehicle should respond to the traffic situation and implements the necessary dynamic driving actions using the steering, accelerator and brakes.

3.2 Technical requirements for the future.

The step to highly automated driving represents a major technical challenge, not least because it represents the end of permanent monitoring by the driver. The system must therefore be capable of dealing with every conceivable fault by itself. This places tremendous and unprecedented demands on the availability and reliability of a highly automated driving system.

High-definition maps allow the forecasting horizon to be extended beyond the range of the sensors. This allows system limitations or situations that a highly automated vehicle – on extremely rare occasions – does not handle perfectly to be recognised at an early stage so that the task of driving can be delegated to the driver again in good time. In addition to this, the reliability and quality of the environmental projection – the 360° environmental sensing in other words – are greatly improved again by the use of high-precision map material. The exact route of traffic lanes can be determined from a map, for instance. Landmarks stored in the map furthermore make it possible to calculate the vehicle’s own position exactly. High-precision maps therefore have an essential role to play in overcoming the enormous challenges of highly automated driving, which is why the BMW Group has acquired a stake in the digital map company HERE, for instance.

Current concept development is addressing the question of the right density of information to be stored in the map. This development work is seeking to strike an optimum balance between the quality and amount of map information on the one hand and, on the other, the quantity and performance of the vehicle sensors used / the intelligence of the algorithms.

4.Fail-operational system.

Delegating the responsibility for controlling the vehicle to the vehicle itself for a certain period of time is due to become permissible by law in Germany before the end of this year, as well as in further countries in the near future. At present, the driver is responsible for the task of driving at all times, even if they are allowed to take their hands off the steering wheel for a few seconds for a substantial strain-relieving effect on long journeys, in particular. While it is true that humans are responsible for most traffic accidents, at the same time they are the best preventers of accidents too. Current systems already work very well, yet they are certainly not yet able to substitute human intelligence in certain situations. And the driver needs to be aware of this.

For this reason, the BMW Group is anxious to make it clear to its customers that the products currently available are driver assistance systems and names them accordingly. Technology has advanced to the stage where we are now on the cusp of highly automated driving. This doesn’t just entail the further development of existing sensor systems, it also calls for a whole new understanding of safety, a stable cloud-based backend and highly dynamic HD map information. This represents a big and extremely challenging technological leap forward. If a vehicle is to temporarily assume responsibility for controlling itself, then we need fail-operational systems, where a fault does not result in failure of the entire system. Brakes, steering and the electrical system that supplies them each require a double safeguard to ensure that the vehicle can continue to be driven in the event of a fault. The BMW Group together with its partners will complete these large-scale tasks by 2021.

5.Autonomous driving: accidentstatistics and ethics.

Highly/fully automated vehicles and autonomous vehicles will play an instrumental role in bringing about a substantial reduction in the total number of accidents in future. During the first few generations of autonomous driving, however, such vehicles will neither have the technical capability to make ethical decisions nor will they be allowed to by constitutional law.

The BMW Group deems the ethical question of whether a vehicle should be capable of taking a life and death decision as being of key importance when it comes to society’s acceptance of autonomous driving. However, the findings from the BMW Group’s accident research provide clear evidence that no guiding examples have been found to date, meaning that such situations are all but non-existent in real-life traffic accidents.

The aim of the autonomous vehicles of the future will be to greatly reduce the probability of accidents compared to road traffic today or prevent accidents from occurring at all by driving in an anticipatory manner. In the unlikely event of being faced with such a dilemma, the technology in the first generations of such vehicles will initially only be capable of recognising whether the driving space ahead is “clear and passable” or “not clear” / “not passable” anyway. In a critical situation, vehicles will be designed to immediately brake at full power. If the vehicle detects that the initiated braking manoeuvre will not suffice to avoid an imminent collision, it will scan for a potential evasive manoeuvre and, if appropriate, change course to move into a free channel. If no suitable driving space is available for performing such an evasive manoeuvre, the existing direction of travel is maintained while braking at maximum power so that any collision takes place at the lowest possible speed. It should furthermore be noted that when travelling at the usual speeds in built-up areas (between 30 and 50 km/h [19 – 31 mph]), the maximum distance the vehicle can swerve to one side is 0.5 to 1.5 metres.

The majority of accidents on our roads today are caused not by exceeding the speed limit but by travelling at an inappropriate speed or driving in an inappropriate manner for the prevailing situation. By using its intelligent connectivity resources, the available real-time services, etc., an autonomous or automated vehicle will have the ability to detect critical situations arising from this at an early stage. It will furthermore be capable of reducing the speed irrespective of the current speed limit and adapting it to suit the situation. Unlike humans, automated vehicles do not get distracted or tired either, and they react consistently in critical situations.

6.The Unterschleißheim campus. 
New development centre for autonomous driving.

At the end of 2016, around 600 employees at the BMW Group were working on the development of highly automated driving. In 2017, the BMW Group is now pooling together all the company’s vehicle connectivity and automated driving expertise at a new campus in Unterschleißheim near Munich.

The new development centre is set to facilitate agile, company-wide collaboration as well as helping to enable high levels of individual decision-making. Once the new facility is fully completed, there will be over 2,000 employees stationed there working on all the developments required for the next steps down the road to fully automated driving – from the software right through to road testing. Alongside the inauguration of the campus, a total of 40 BMW 7 Series test vehicles for highly and fully automated driving on motorways and in urban environments will be built in 2017 and trials started. These vehicles will be put into operation at Intel (USA), Mobileye (Israel) and BMW Group (Munich) facilities.

7.Vehicles in development.
40 automated BMW 7 Series cars by the end of 2017.

The CES 2017 fair in Las Vegas saw the BMW Group announce its intention to release a succession of prototype vehicles during 2017 in collaboration with Intel and Mobileye. These will form a fleet of 40 highly automated and fully automated vehicles by the end of the year. Test drives will take place on public roads and focus on two main types of use: driving without oncoming traffic (motorways) and driving in city centre environments. The test drives will be mainly conducted in the home countries of the three partners, namely USA, Israel and Germany.

By developing these BMW 7 Series advanced prototypes collaboratively, the partners will ensure the timely roll out of the BMW Group's first highly-automated series vehicle (level 3) – the BMW iNext, due in 2021. BMW iNext is the BMW Group's first venture into highly-automated driving. From a technical perspective, the BMW iNext will also be capable of level 4 and 5 operation. Whether or not this is achievable in practice depends on a number of external factors, but it is not yet possible to predict how these will develop.

For an autonomous vehicle to be considered market ready, it must behave safely and reliably in any conceivable driving situation, as well as operating in a way that is predictable for other road users. Theoretical calculations have determined that around 240 million kilometres (150 million miles) of testing on public roads would be needed to provide assurance for every situation. In practice, this is neither practicable nor sensible. In fact, the most relevant tests relate to a much smaller number of critical driving situations, not the total distance travelled. Instead, autonomous vehicle safeguarding is carried out by analysing "foundation" situations that have been investigated in real-world trials. These situations are then extrapolated using stochastic simulation to provide comprehensive validation. For example, in future BMW will be in a position where it is able to test around five million driving situations per simulation for every software release within a very short space of time.

8.Artificial intelligence for autonomousdriving.

Artificial intelligence is a discipline within the field of computer science. Its goal is to use computer programs to solve problems that could not otherwise be solved without using the intelligence of a human being. Artificial intelligence is important as a key technology for many aspects of mobility, now and in the future.

There are many different areas at BMW where it is being applied. These include optimisation of production processes and the development of customised natural-language interactions for customers. Another field where artificial intelligence can be applied is in the creation of highly accurate road maps with dynamic content, such as temporary obstacles and live traffic information. It can also play a key role in intelligent multimodal routing, intelligent car sharing and ride sharing, provision of location-based services and other services that are personalised based on user context.

The BMW Group is already active in all these areas and is working on combining them into an total user experience that is both attractive and useful to users.

Artificial intelligence is increasingly allowing computers to find solutions to highly complex problems, something that would have been inconceivable just a few years ago. Software developers at the BMW Group are playing a significant role in such developments and have the opportunity to experience the new technology directly through the product.

Artificial intelligence as a key enabler for autonomous driving.
It was clear even in the early days that autonomous driving would not become a reality if purely rules-based approaches were used. Instead, realising the vision of autonomous driving requires machine learning systems.

A diverse range of real-world data must be collected by a vehicle's on-board sensors in order to facilitate a data-driven development cycle. This results in vast quantities of data that must then be processed and made available by the artificial intelligence system. A data centre is currently being set up for this purpose, in collaboration with Intel, and will be further extended in the coming months. Training of neural networks and further development of algorithms requires the data to be always quickly accessible, so the facility is being equipped with a corresponding amount of computing power. The data centre will also simulate scenarios that occur so rarely in the real world that test coverage could not otherwise be truly comprehensive.

The result is artificial intelligence with an ever-increasing ability to develop models of reality.

Another artificial intelligence system is required in the vehicle to make an intelligent interpretation of the situations it faces based on the models. Without this, the vehicle cannot derive a driving strategy with the necessary degree of confidence.

However, a whole range of challenges associated with machine learning must be overcome before it is suitable for series applications.

These include:
Data

– Globally coordinated logging
– Central data storage
– Labelling
– Long-term usability of data

Expertise
– Application-oriented modelling of neural networks
– Parameterisation of learning methods
– Dealing with large quantities of data

Hardware
– High performance computing infrastructure for training
– Powerful in-vehicle computing platform
– Backend connection for updates and feedback loops

Safeguarding 
– Generalisation for completely new situations
– Safe degradation
– Dealing with rare hazardous situations

9.Partnerships and contributions.

The BMW Group is following a clearly defined strategy for autonomous driving and has identified three key technology elements:
- High definition (HD) live mapping.
- High-performance sensors, a supercomputer and intelligent software. These are needed for reliable acquisition and real-time processing of informationa bout the environment and for making safe decisions relating to manoeuvres that are similar to those a human would reach.
- Total integration of a system in the vehicle that is safe, secure and has high availability.

Participation in HERE.
Highly accurate, continuously updated maps are already playing a key role in the development of highly automated driving. This is the reason that the BMW Group, together with Audi AG and Daimler AG, pursued a successful purchase of the HERE mapping business from Nokia in December 2015. HERE is one of the leading technology providers in the navigation data sector. One of the most important goals here was to establish and develop a leading data ecosystem using location-based services. The underlying HERE location platform will be openly accessible to all relevant players in the market. Changes have been made to the ownership and governance structure at HERE in order to ensure that the company retains its independence and is not influenced in the way the business operates.

The location platform developed by HERE combines high-definition maps with location-based, real-time traffic information to provide the user with a detailed representation of the real world that is accurate to the second. This platform is based on HERE's industry-leading mapping technology and draws on information from a wide range of data sources, including vehicles, mobile phones, the transport and logistics sector and even infrastructure. In future, the plan is for data delivered by the sensors on several million vehicles to be combined to form a single data pool, which will help accelerate the development of a shared location platform. The goal is to obtain even more precise information about a vehicle's environment. This will deliver enormous benefit to all HERE customers in the form of improved comfort while driving and travelling, greater traffic safety and fewer traffic jams. This in turn will ultimately result in reduced emissions and cleaner towns and cities. BMW is already providing anonymised sensor data relating to traffic information and road signs. The next phase of development, to allow updating of HD maps via the BMW fleet, is now almost complete.

HERE is continuously expanding the scope of its business activities. As well as maintaining a market-leading position in the automotive sector, the company also intends to intensify its activities in the consumer and enterprise sectors. To this end, HERE will boost its client acquisition activities outside the automotive industry.

Collaboration with Intel and Mobileye. In July 2016, the BMW Group, Intel and Mobileye announced a wide-ranging collaboration. They are combining their strengths to realise the vision of self-driving vehicles and to accelerate the development of cutting-edge mobility concepts, which hold great promise for the future.

Since the start of this collaboration, the three companies have developed a scalable architecture that can be adapted by other manufacturers and developers so that they can pursue their own design objectives and achieve differentiation between brands. This non-exclusive platform offers an ecosystem for the development of autonomous driving. It covers essential on-board elements, including concepts for sensors, functionality, safety and security, as well as the functional software, which comprises the environment model anddriving strategy. Beyond the vehicle, users such as OEMs and tier 1 suppliers will gain a consistent data management toolchain and a high-performance simulation package, giving them everything they need to implement high-quality automated driving functions that are safe and secure.

Intel's contribution to the partnership is its innovative, high-performance computing solutions, which find applications everywhere from the vehicle to the data centre. What's more, Intel's world-leading processors and FPGA technologies can deliver the most efficient balance of processing speed and capacity, while still satisfying the stringent demands of the automotive industry in terms of heat development and safety.

Mobileye brings to the table its patented EyeQ®5 high-performance computer vision processor, which offers world-leading image processing technology operating at the highest levels of energy efficiency and safety. The EyeQ®5 is designed for processing and interpretation of an eight-way camera system, which provides a 360° view – something that is especially important in urban applications. Combined with Intel CPUs (central processing units) and Altera FPGAs, the result is a central supercomputing platform that is suitable for automotive applications from level 3 to level 5.

The BMW Group and Mobileye are jointly developing related solutions in the field of sensor data fusion, in order to provide a comprehensive model of the vehicle environment based on input from radar, lidar, ultrasound and camera sensors. A driving policy based on artificial intelligence is also being developed to help with mastering the infinite number of complex driving situations.

REMTM (Mobileye Road Experience Management) data collection technology will be linked with HERE-based backend technology and used in all new BMW models with launch dates in 2018. This decision by the two partners – the BMW Group and Mobileye – represents the starting point for a continuously expanding vehicle fleet that will facilitate real-time crowd-sourced collection of data via camera-based advanced driver assistance systems (ADAS). At the same time, it marks a major milestone for autonomous driving using high-resolution (HD) maps, which are designed to make driving even safer and more efficient.

The BMW Group's responsibility in these pioneering partnerships is to develop the core functions and the testing and safeguarding environment, including simulation. Their contribution is also aligned to the BMW Group's business goals. BMW places particular importance on the design of the safety concept. This is because the company wants to provide other platform users with the best possible starting point for their own implementations, as well as establishing essential confidence in the platform as it is developed.

Further information:
Press release from the BMW Group, Intel and Mobileye (July 2016). 
BMW Group, Intel and Mobileye team up to bring fully autonomous driving to streets by 2021.
(https://www.press.bmwgroup.com/deutschland/article/detail/T0261586EN/bmw-group-intel-and-mobileye-team-up-to-bring-fully-autonomous-driving-to-streets-by-2021)

Press release from the BMW Group, Intel and Mobileye (January 2017).
BMW Group, Intel and Mobileye will have autonomous test vehicles on the roads by the second half of 2017. (https://www.press.bmwgroup.com/usa/article/detail/T0266995EN_US/bmw-group-intel-and-mobileye-will-have-autonomous-test-vehicles-on-the-roads-by-the-second-half-of-2017?language=en_US)

Press release from the BMW Group and Mobileye (February 2017).
Crowd sourcing for automated driving: BMW Group and Mobileye agree to generate new kind of sensor data. (http://s2.q4cdn.com/670976801/files/doc_news/Crowd-sourcing-for-automated-driving-BMW-Group-and-Mobileye-agree-to-generate-new-kind-of-sensor-data.pdf)

In the event of enquiries please contact:
Manfred Poschenrieder, Comunications Autonomous Driving
E-Mail: manfred.poschenrieder@bmwgroup.de

Cypselus Von Frankenberg
Head of BMW Group Innovation and Design Communication 
Tel.: +49-89-382-30641, E-Mail: Cypselus.von-Frankenberg@bmw.de

The BMW Group
With its four brands BMW, MINI, Rolls-Royce and BMW Motorrad, the BMW Group is the world’s leading premium manufacturer of automobiles and motorcycles and also provides premium financial and mobility services. As a global company, the BMW Group operates 31 production and assembly facilities in 14 countries and has a global sales network in more than 140 countries.

In 2016, the BMW Group sold approximately 2.367 million cars and 145,000 motorcycles worldwide. The profit before tax was approximately € 9.67 billion on revenues amounting to € 94.16 billion. As of 31 December 2016, the BMW Group had a workforce of 124,729 employees.

The success of the BMW Group has always been based on long-term thinking and responsible action. The company has therefore established ecological and social sustainability throughout the value chain, comprehensive product responsibility and a clear commitment to conserving resources as an integral part of its strategy.

www.bmwgroup.com
Facebook: http://www.facebook.com/BMWGroup
Twitter: http://twitter.com/BMWGroup
YouTube: http://www.youtube.com/BMWGroupview
Google+: http://googleplus.bmwgroup.com