AUTONOMOUS VEHICLE TECHNOLOGY REPORT - 2020 The guide to understanding the current state of the art in hardware & software for self-driving ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
2020
AUTONOMOUS
VEHICLE
TECHNOLOGY
REPORT The guide to understanding
the current state of the art
in hardware & software for
self-driving vehicles.
sponsored by
For the people who aim to create a better version of the future.
Contributors 6 Introduction 10 How this report came to be: a collaborative effort 11 Levels of Autonomy 14 Sensing 17 Environmental mapping 18 Passive sensors 18 Active Sensors 22 Choice of Sensors 28 Geolocalization 32 Maps 33 Thinking & Learning 35 SLAM and Sensor Fusion 35 Machine Learning Methods 38 Gathering Data 40 Path Planning 42 Acting 44 Architectures: Distributed versus Centralized 45 Power, Heat, Weight, and Size challenges 47 User experience 48 Inside the vehicle 51 The external experience 53 Communication & Connectivity 55 DSRC or C-V2X 59 Use case: Autonomous Racing 62 Summary 66 About Nexperia 68 About Wevolver 70 References 72
Contributors
Editor in Chief Jordan Sotudeh Akbar Ladak Joakim Svennson
Cover Photographer Many thanks to
Bram Geenen Los Angeles, USA Bangalore, India Norrköping, Sweden Benedict Redgrove The people at Roborace, specifically Victo-
Amsterdam, The Netherlands Senior Strategic Analyst at NASA Jet Pro- Founder, CEO, Kaaenaat, which develops Senior ADAS Engineer, Function Owner London, United Kingdom ria Tomlinson and Alan Cocks.
CEO Wevolver. pulsion Laboratory. autonomous robots for logistics, retail and Traffic Sign Recognition and Traffic Light Benedict has a lifelong fascination with
Master International Science and Technol- security use cases, as well as Advanced Recognition at Veoneer. technology, engineering, innovation and Edwin van de Merbel, Dirk Wittdorf, Petra
ogy Policy, Elliott School of International Driver Assistance Systems (ADAS) for 2- & MSc. Media Technology, Linköping Univer- industry, and is a dedicated proponent of Beekmans - van Zijll and all the other
Editors Affairs, Washington DC, USA. 4- wheeler vehicles for chaotic driving sity, Sweden. modernism. This has intuitively led him people at Nexperia for their support.
conditions in Asia & Africa. to capturing projects and objects at their
Ali Nasseri Matthew Nancekievill Master in Electrical Engineering, Georgia Fazal Chaudry most cutting edge. He has created an aes- Our team at Wevolver; including Sander
Vancouver, Canada Manchester, United Kingdom Institute of Technology. Headington, United Kingdom thetic of photography that is clean, pure Arts, Benjamin Carothers, Seth Nuzum,
Lab manager at the Programming Lan- Postdoctoral researcher submersible ro- Product Development Engineer. and devoid of any miscellaneous informa- Isidro Garcia, Jay Mapalad, and Richard
guages for Artificial Intelligence (PLAI) botics, University of Manchester, UK. Zeljko Medenica Master of Science, Space Studies, Interna- tion, winning him acclaim and numerous Hulskes. Many thanks for the proofreads
research group at the University of British PhD. Electrical and Electronics Engineer- Birmingham, Michigan, USA tional Space University, Illkirch Graffensta- awards. and feedback.
Columbia. ing, University of Manchester, UK. Principal Engineer and Human Machine den, France. Redgrove has amassed a following and
Previously Chair of the Space Generation CEO Ice Nine Robotics. Interface (HMI) Team Lead at the US client base from some of the most ad- The Wevolver community for their support,
Advisory Council. R&D Center of Changan, a major Chinese Shlomit Hacohen vanced companies in the world. A career knowledge sharing, and for making us
Cum Laude PhD. in Engineering Physics, Jeremy Horne automobile manufacturer. Previously led Tel Aviv, Israel spent recording the pioneering technol- create this report.
Politecnico di Torino. San Felipe, Baja California, Mexico research on novel and intuitive HMI for VP of Marketing at Arbe Robotics; develop- ogy of human endeavours has produce a
President Emeritus of the American Asso- Advanced Driver Assistance Systems at ing ultra high-resolution 4D imaging radar photographic art form that gives viewers a Many others that can’t all be listed here
Adriaan Schiphorst ciation for the Advancement of Science, Honda. technology. window into an often unseen world, such have helped us in big or small ways. Thank
Amsterdam, The Netherlands Southwest Division. PhD. in Electrical and Computer Engineer- MBA at Technion, the Israel Institute of as Lockheed Martin Skunk Works, UK MoD, you all.
Technology journalist. Science advisor and curriculum coordina- ing from the University of New Hampshire. Technology European Space Agency, British Aerospace
MSc Advanced Matter & Energy Physics at tor at the Inventors Assistance Center. and NASA. Whether capturing the U-2 re- Beyond the people mentioned here we
University of Amsterdam and the California Ph.D. in philosophy from the University of Maxime Flament Designer connaissance pilots and stealth planes, the owe greatly to the researchers, engineers,
Institute of Technology. Florida, USA. Brussels, Belgium Navy Bomb Disposal Division or spending writers, and many others who share their
Previously editor at Amsterdam Science Chief Technology Officer, 5G Automotive Bureau Merkwaardig time documenting the NASA’s past, present knowledge online. Find their input in the
Journal. Drue Freeman Association (5GAA) Amsterdam, The Netherlands and future, Benedict strives to capture references.
Cupertino, California, USA PhD. in Wireless Communication Systems, Award winning designers Anouk de the scope and scale of advancements and
Contributing Experts CEO of the Association for Corporate Chalmers University of Technology, Göte- l’Ecluse and Daphne de Vries are a creative what they mean to us as human beings. Media Partner
Growth, Silicon Valley. borg, Sweden. duo based in Amsterdam. They are special- His many awards include the 2009 AOP
Norman di Palo Former Sr. Vice President of Global ized in visualizing the core of an artistic Silver, DCMS Best of British Creatives, and Supplyframe
Rome, Italy Automotive Sales & Marketing for NXP Mark A. Crawford Jr. problem. Bureau Merkwaardig initiates, the Creative Review Photography Annual Supplyframe is a network for electronics
Robotics and Machine Learning researcher, Semiconductors. Baoding City, China develops and designs. 2003, 2008, and 2009. design and manufacturing. The company
conducting research on machine learning Board Director at Sand Hill Angels. Adviso- Chief Engineer for Autonomous Driving At Wevolver we are a great fan of Bene- provides open access to the world’s largest
for computer vision and control at the Isti- ry Board Member of automotive companies Systems at Great Wall Motor Co. Illustrations dict’s work and how his pictures capture a collection of vertical search engines,
tuto Italiano di Tecnologia, Genova, Italia. Savari and Ridar Systems, and Advisory PhD. Industrial and Systems Engineering spirit of innovation. We’re grateful he has supply chain tools, and online communi-
Cum Laude MSc. Engineering in Artifi- Board Member of Silicon Catalyst, a semi- - Global Executive Track, at Wayne State Sabina Begović enabled us to use his beautiful images of ties for engineering. Their mission is to
cial Intelligence and Robotics, Sapienza conductor focused incubator. University. Padua, Italy the Robocar to form the perfect backdrop organize the world of engineering knowl-
Università di Roma, and graduate of the Pi Bachelor of Science in Electrical Engineer- Previously Technical Expert at Ford. Croation born Sabina is a visual and inter- for this report. edge to help people build better hardware
School of Artificial Intelligence. ing, San Diego State University, MBA from action designer. She obtained a Master’s in products, and at Wevolver we support that
Pepperdine University, Los Angeles. William Morris Visual and Communication Design at Iuav, aspiration and greatly appreciate that Sup-
Detroit, Michigan, USA University of Venice, and a Masters in Art plyframe contributes to the distribution of
Automotive engineer Education at the Academy of Applied Art, this report among their network.
Rijeka, Croatia.
6 7
“It’s been an enormously
difficult, complicated
slog, and it’s far more
complicated and involved
than we thought it would
be, but it is a huge deal.”
Nathaniel Fairfield,
distinguished software
engineer and leader of the
‘behavior team’ at Waymo,
December 2019 [1]
8 9
Introduction
Bram Geenen Motorized transportation has changed Therefore, this report’s purpose is to How this report This report would not have been
Editor in Chief, the way we live. Autonomous vehicles enable you to be up to date and un- possible without the sponsorship of
CEO of Wevolver are about to do so once more. This derstand autonomous vehicles from a came to be: Nexperia, a semiconductor company
evolution of our transport - from hors- technical viewpoint. shipping over 90Bn components an-
es and carriages, to cars, to driverless We have compiled and centralized the a collaborative nually, the majority of which are with-
vehicles, - has been driven by both information you need to understand in the automotive industry. Through
technical innovation and socioeco- what technologies are needed to effort their support, Nexperia shows a
nomic factors. In this report we focus develop autonomous vehicles. We will commitment to the sharing of objec-
on the technological aspect. elaborate on the engineering consid- Once the decision was made to create tive knowledge to help technology
erations that have been and will be this report, we asked our communi- developers innovate. This is the core
Looking at the state of autonomous made for the implementation of these ty for writers with expertise in the of what we do at Wevolver.
vehicles at the start of the 2020s we technologies, and we’ll discuss the field, and for other experts who could
can see that impressive milestones current state of the art in the industry. provide input. A team of writers and The positive impact these technol-
have been achieved, such as compa- editors crafted a first draft, leveraging ogies could possibly have on both
nies like Waymo, Aptiv, and Yandex This reports’ approach is to describe many external references. Then, in a individual lives, and our society and
offering autonomous taxis in dedicat- technologies at a high level, to offer second call-out to our community we planet as a whole are an inspiring
ed areas since mid-2018. At the same the baseline knowledge you need to found many engineers and leaders and worthwhile goal. At Wevolver
time, technology developers have run acquire, and to use lots of references from both commercial and academic we hope this report provides the
into unforeseen challenges. to help you dive deeper whenever backgrounds willing to contribute information and inspiration for you in
needed. significant amounts of their time any way possible to be a part of that
Some industry leaders and experts and attention to providing extensive evolution.
have scaled back their expectations, Most of the examples in the report feedback and collaborating with us
and others have spoken out against will come from cars. However, indi- to shape the current report through
optimistic beliefs and predictions.[2,3] vidual personal transportation is not many iterations. We owe much to
Gartner, a global research and advi- the only area in which Autonomous their dedication, and through their
sory firm, weighs in by now placing Vehicles (AVs) will be deployed and input this report has been able to
‘autonomous vehicles’ in the Trough of in which they will have a significant incorporate views from across the
Disillusionment of their yearly Hype impact. Other areas include public industry and 11 different countries.
Cycle.[4] transportation, delivery & cargo and
specialty vehicles for farming and Because this field continues to
The engineering community is less mining. All of these come with their advance, we don’t consider our work
affected by media hype: Over 22% of own environment and specific usage done. We intend to update this report
the engineers visiting the Wevolver requirements that are shaping AV into new editions regularly as new
platform do so to gain more knowl- technology. At the same time, all of knowledge comes available and our
edge on autonomous vehicle technol- the technologies described in this re- understanding of the topic grows.
ogy.[5] Despite how much topics like port form the ingredients for autono- You are invited to play an active role
market size and startup valuations my, and thus will be needed in various and contribute to this evolution, be it
have been covered globally by the applications. through brief feedback or by submit-
media, many engineers have ex- ting significant new information and
pressed to our team at Wevolver that insights to our editorial team (info@
comprehensive knowledge to grasp wevolver.com), your input is highly
the current technical possibilities is appreciated and invaluable to further
still lacking. the knowledge on this topic.
10 11
“Autonomous vehicles
are already here – they’re
just not very evenly
distributed.”
William Gibson,
Science fiction writer,
April 2019 [12]
12 13
Levels of Autonomy
When talking about autonomous ve- Level 0 (L0): Level 2 (L2): Level 3 (L3): Level 4 (L4): Level 5 (L5):
hicles, it is important to keep in mind No automation Now both steering and accelera- Conditional automation: The sys- These systems have high auto- Full automation, the vehicle can
that each vehicle can have a range of tion are simultaneously handled tem can drive without the need mation and can fully drive them- drive wherever, whenever.
autonomous capabilities. To enable Level 1 (L1): by the autonomous system. The for a human to monitor and re- selves under certain conditions.
classification of autonomous vehicles, Advanced Driver Assistance Sys- human driver still monitors the spond. However, the system might The vehicle won’t drive if not all
the Society Of Automotive Engineers tems (ADAS) are introduced: fea- environment and supervises the ask a human to intervene, so the conditions are met.
(SAE) International established its tures that either control steering support functions. driver must be able to take con-
SAE J3016™ “Levels of Automated or speed to support the driver. For trol at all times.
Driving” standard. Its levels range example, adaptive cruise control
from 0-5 and a higher number des- that automatically accelerates and
ignates an increase in autonomous decelerates based on other vehi-
capabilities.[6] cles on the road.
Levels of driving automation summary.
Adapted from SAE by Wevolver. [6]
ZZ Z ZZ Z
Z ZZ
0 00 1 11 2 22 3 33 4 44 5 55
NO AUTOMATION NO
NOAUTOMATION
AUTOMATION
DRIVER ASSISTANCE
DRIVER
DRIVERASSISTANCE
ASSISTANCE
PARTIAL AUTOMATION
PARTIAL
PARTIALCONDITIONAL
AUTOMATION
AUTOMATIONAUTOMATION
CONDITIONAL
CONDITIONALAUTOMATION
AUTOMATION
HIGH AUTOMATIONHIGH
HIGHAUTOMATION
AUTOMATION
FULL AUTOMATIONFULL
FULLAUTOMATION
AUTOMATION
You monitor the environment.
You
Youmonitor
monitor
Youthe
the
areenvironment.
environment.
the driver, You
Youare
arethe
thedriver,
When
driver,system requests,
When
Whensystem
systemrequests,
requests,
even when automationeven
even
features
when
whenare
automation
automation
turned on.
features
featuresare
areturned
turned
you
on.
on.must take control.
you
youmust
musttake
takecontrol.
control.
No requirement for you
No
Noto
requirement
requirement
take over control.
for
foryou
youtototake
takeover
overcontrol.
control.
System operates whenSystem
System
specific
operates
operateswhen
whenspecific
specific System operates in allSystem
Systemoperates
operatesininall
all
System suports you driving.
System
Systemsuports
suportsyou
youdriving.
driving. conditions are met. conditions
conditionsare
aremet.
met. conditions conditions
conditions
Steering OR speed Steering
SteeringOR
ORspeed
speed
are automated. are
areautomated.
automated. Steering AND speed are
Steering
automated.
Steering ANDspeed
AND speedare
areautomated.
automated.
14 15
The context and environment (in-
cluding rules, culture, weather, etc.)
in which an autonomous vehicle
Level 5 ADS have the same mobility
as a human driver: an unlimited ODD.
Designing the autonomous vehicle to
Sensing
needs to operate greatly influences be able to adjust to all driving sce-
the level of autonomy that can be narios, in all road, weather and traffic
achieved. On a German Autobahn, the conditions is the biggest technical Because an autonomous vehicle oper-
speed and accuracy of obstacle de- challenge to achieve. Humans have ates in an (at least partially) unknown
tection, and the subsequent decisions the capability to perceive a large and dynamic environment, it simulta-
that need to be made to change the amount of sense information and neously needs to build a map of this
speed and direction of the vehicle fuse this data to make decisions us- environment and localize itself within
need to happen within a few milli- ing both past experience and our im- the map. The input to perform this
seconds, while the same detection agination. All of this in milliseconds. Simultaneous Localization and Map-
and decisions can be much slower A fully autonomous system needs to ping (SLAM) process needs to come
for a vehicle that never leaves a match (and outperform) us in these from sensors and pre-existing maps
corporate campus. In a similar matter, capabilities. The question of how to created by AI systems and humans.
the models needed to drive in sunny assess the safety of such a system
Arizona are more predictable than needs to be addressed by legislators.
those in New York City, or Banga- Companies have banded together,
Static Moving Road Lane Traffic Street
lore. That also means an automated like in the Automated Vehicle Safety Objects Objects Markings Markings Lights Signs
driving system (ADS) capable of L3 Consortium, to jointly develop new
automation in the usual circumstanc- frameworks for safety.[10]
es of e.g. Silicon Valley, might need
to fall back to L2 functionality if it Major automotive manufacturers,
would be deployed on snowy roads as well as new entrants like Google
or in a different country. (Waymo), Uber, and many startups
are working on AVs. While design
The capabilities of an autonomous concepts differ, all these vehicles rely
vehicle determine its Operational on using a set of sensors to perceive
Design Domain (ODD). The ODD the environment, advanced software
defines the conditions under which to process input and decide the
a vehicle is designed to function and vehicle’s path and a set of actuators
is expected to perform safely. The to act upon decisions. [11] The next
ODD includes (but isn’t limited to) sections will review the technologies
environmental, geographical, and needed for these building blocks of
time-of-day restrictions, as well as autonomy.
traffic or roadway characteristics.
For example, an autonomous freight
truck might be designed to transport
cargo from a seaport to a distribu-
tion center 30 Km away, via a specific
route, in day-time only. This vehicles
ODD is limited to the prescribed
route and time-of-day, and it should
not operate outside of it.[7–9]
Example of the variety of static and
moving objects that an autonomous
vehicle needs to detect and distinguish
from each other. Image: Wevolver,
based on a photo by Dan Smedley.
16 17
Environmental Passive sensors This leads to higher noise susceptibil-
ity for CMOS sensors, such that CCD
mapping Due to the widespread use of object
detection in digital images and vide-
sensors can create higher quality im-
ages. Yet, CMOS sensors use up to 100
os, passive sensors based on camera times less power than CCDs. Further- LIDAR Ir Cameras
In order to perceive a vehicle’s direct technology were one of the first more, they’re easier to fabricate using
environment, object detection sensors sensors to be used on autonomous standard silicon production processes.
are used. Here, we will make a dis- vehicles. Digital cameras rely on CCD
tinction between two sets of sensors: (charge-coupled device) or CMOS Most current sensors used for auton-
passive and active. Passive sensors (complementary metal-oxide semi- omous vehicles are CMOS based and
GNSS Long Range RADAR
detect existing energy, like light or conductor) image sensors which work have a 1-2 megapixel resolution.[15]
radiation, reflecting from objects in by changing the signal received in the
the environment, while active sensors 400-1100 nm wavelengths (visible to While passive CMOS sensors are
send their own electromagnetic near infrared spectra) to an electric generally used in the visual light
signal and sense its reflection. These signal.[13,14] spectrum, the same CMOS technology
Short / Medium
sensors are already found in automo- could be used in thermal imaging IMU Range RADAR
tive products at Level 1 or 2, e.g. for The surface of the sensor is broken cameras which work in the infrared
lane keeping assistance. down into pixels, each of which can wavelengths of 780 nm to 1 mm.
sense the intensity of the signal They are useful sensors for detection
received, based on the amount of of hot bodies, such as pedestrians or
charge accumulated at that location. animals, and for peak illumination
By using multiple sensors that are situations such as the end of a tunnel, Cameras Ultrasound
sensitive to different wavelengths of where a visual sensor will be blinded
light, color information can also be by the light intensity.[16]
encoded in such a system.
In most cases, the passive sensor
While the principle of operation of suite aboard the vehicle consists of
CCD and CMOS sensors are similar, more than one sensor pointing in the
their actual operation differs. CCD same direction. These stereo camer-
sensors transport charge to a specific as can take 3D images of objects by
corner of the chip for reading, while overlaying the images from the differ-
each pixel in a CMOS chip has its own ent sensors. Stereoscopic images can
transistor to read the interaction with then be used for range finding, which
light. Colocation of transistors with is important for autonomous vehicle
sensor elements in CMOS reduces its application.
light sensitivity, as the effective sur-
face area of the sensor for interaction
with the light is reduced.
An example of typical sensors used to perceive the environment. Note that various vehicle
manufacturers may use different combinations of sensors and might use all of the displayed
sensors. For example, increasingly multiple smaller LIDAR sensors are being used, and long
range backward facing RADAR can be incorporated to cover situations like highway lane
changing and merging. The placing of the sensors can vary as well. Image: Wevolver
18 19The main benefits of passive sensors Indeed, Tesla cars mount an array of be done by using a rotating camera “Once you solve cameras for vision, autonomy is
are[17]: cameras all around the vehicle to that takes images at specific inter-
gather visual field information, and vals, or by stitching the images of 4-6
solved; if you don’t solve vision, it’s not solved
• High-resolution in pixels and London based startup Wayve claims cameras together through software. … You can absolutely be superhuman with just
color across the full width of its that its cars which only rely on pas- In addition, these sensors need a high
field of view. sive optic sensors are safe enough for dynamic range (the ability to image cameras.”
• Constant ‘frame-rate’ across the use in cities. The main shortcoming of both highlights and dark shadows in a
field of view. passive sensors is their performance scene), of more than 100 dB,[22] giving Elon Musk,
• Two cameras can generate a 3D in low light or poor weather condi- them the ability to work in various 2017 [19]
stereoscopic view. tions; due to the fact that they do not light conditions and distinguish be-
• Lack of transmitting source re- have their own transmission source tween various objects.
duces the likelihood of interfer- they cannot easily adapt to these
ence from another vehicle. conditions. These sensors also gen- Dynamic range is measured in decibel
• Low cost due to matured tech- erate 0.5-3.5 Gbps of data,[18] which (dB); a logarithmic way of describing
nology. can be a lot for onboard processing a ratio. Humans have a dynamic range “At the moment, LIDAR lacks the capabilities to
• The images generated by these or communicating to the cloud. It is of about 200 dB. That means that in a
systems are easy for users to also more than the amount of data single scene, the human eye can per- exceed the capabilities of the latest technology in
understand and interact with generated by active sensors. ceive tones that are about 1,000,000 radar and cameras.”
times darker than the brightest ones.
If a passive camera sensor suite Cameras have a narrower dynamic
Tetsuya Iijima,
is used on board an autonomous range, though are getting better.
vehicle, it will likely need to see the
General Manager of Advanced Technology De-
whole surrounding of the car. This can velopment for Automated Driving, Nissan,
May 2019 [20]
“Let’s be candid, LIDAR is unaffordable in consumer
vehicles, but if a lidar unit were available today
that had good performance and was affordable, it
would quietly show up in a Tesla car and this whole
Gamma-Ray X-Ray UV Visible IR Microwave Radio hubbub would go away.”
Bill Colleran,
CEO, Lumotive,
June 2019 [21]
10-12 10-10 10-8 10-6 10-4 10-2 100 102 104 106
Wavelength, λ (m)
RADAR
THERMAL CAMERAS
LIDAR
CAMERAS
The electromagnetic spectrum and its
usage for perception sensors .[16]
20 21Active Sensors Ultrasonic sensors (also referred to as RADAR (RAdio Detection And Rang- Time of flight principle, illustrated.
Image: Wevolver.
SONAR; SOund NAvigation Ranging) ing) uses radio waves for ranging.
Active sensors have a signal transmis- use ultrasound waves for ranging and Radio waves travel at the speed of The distance can be calculated using the
sion source and rely on the principle are by far the oldest and lowest cost light and have the lowest frequency formula d=(v⋅t)/2. Where d is the distance,
of these systems. As sound waves (longest wavelength) of the electro- v is the speed of the signal (the speed of
environment. ToF measures the travel have the lowest frequency (longest magnetic spectrum. RADAR signals sound for sound waves, and the speed of
time of a signal from its source to a wavelengths) among the sensors - light for electromagnetic waves) and t is
used, they are more easily disturbed. rials that have considerable electrical the time for the signal to go to reach the
object and reflect back. This calculation
the signal to return. This means the sensor is easily conductivity, such as metallic objects.
method is the most common but has lim-
affected by adverse environmental Interference from other radio waves itations and more complex methods have
The frequency of the signal used de- conditions like rain and dust. Inter- can affect RADAR performance, while been developed; for example, using the
termines the energy used by the sys- ference created by other sound waves transmitted signals can easily bounce phase-shift in a returning wave. [23]
tem, as well as its accuracy. Therefore, can affect the sensor performance off curved surfaces, and thus the
determining the correct wavelength as well and needs to be mitigated by sensor can be blind to such objects.
plays a key role in choosing which using multiple sensors and relying on At the same time, using the bouncing
system to use. additional sensor types. In addition, as properties of the radio waves can
sound waves lose energy as distance enable a RADAR sensor to ‘see’ beyond
increases, this sensor is only effective objects in front of it. RADAR has less-
over short distances such as in park er abilities in determining the shape
assistance. More recent versions rely of detected objects than LIDAR.[25]
on higher frequencies, to reduce the Signal in
likelihood of interference.[24] -
DAR are its maturity, low cost, and
resilience against low light and bad
weather conditions. However, radar
can only detect objects with low
spatial resolution and without much
information about the spatial shape
of the object, thus distinguishing
between multiple objects or separat-
ing objects by direction of arrival can
be hard. This has relegated radars to
more of a supporting role in automo- Signal out
tive sensor suites.[17]
Di
sta
nc
em
“We need more time for the car to re- ea
su
act, and we think imaging radar will be re d
a key to that.”
Chris Jacobs, Vice President of Autonomous Transporta-
tion and Automotive Safety, Analog Devices Inc,
January 2019 [26]
22 23“Almost everything is in R&D, of which 95 per- Imaging radar is particularly interest- LIDAR Systems that do not use any
ing for autonomous cars. Unlike short mechanical parts are referred to as
cent is in the earlier stages of research, rather range radar which relies on 24GHz ra- solid-state, and sometimes as ‘LIDAR-
than actual development, the development stage dio waves, imaging radar uses higher on-a-chip.’
energy 77-79 GHz waves. This allows
is a huge undertaking — to actually move it to- the radar to scan a 100 degree field Flash LIDARS are a type of solid-state
wards real-world adoption and into true series of view for up to a 300 m distance. LIDARS that diffuse their laser beam
This technology eliminates former to illuminate an entire scene in one
production vehicles. Whoever is able to enable resolution limitations and generates flash. The returning light is captured
true autonomy in production vehicles first is go- a true 4D radar image of ultra-high by a grid of tiny sensors. A major chal-
resolution.[15,26,27] lenge of Flash LIDARS is accuracy.[30]
ing to be the game changer for the industry. But
that hasn’t happened yet.” LIDAR (LIght Detection And Ranging)
uses light in the form of a pulsed
Phased-Array LIDARS are another
solid-state technology that is under-
laser. LIDAR sensors send out 50,000 going development. Such systems
Austin Russell, founder and CEO of - 200,000 pulses per second to cover feed their laser beam into a row of
Luminar, June 2019 [21] an area and compile the returning emitters that can change the speed
signals into a 3D point cloud. By and phase of the light that passes
comparing the difference in consec- through.[31] The laser beam gets
utive perceived point clouds, objects pointed by incrementally adjusting
and their movement can be detected the signal’s phase from one emitter to
such that a 3D map, of up to 250m in the next.
range, can be created.[28]
Metamaterials: A relatively new
There are multiple approaches to development is to direct the laser by
LIDAR technology: shining it onto dynamically tunable
metamaterials. Tiny components on
Mechanical scanning LIDARS use these artificially structured metas-
rotating mirrors and/or mechanically urfaces can be dynamically tuned to
rotate the laser. This setup provides a slow down parts of the laser beam,
wide Field Of Vision but is also rela- which through interference results
tively large and costly. This technolo- in a beam that’s pointing in a new
gy is the most mature. direction. Lumotive, a startup funded
by Bill Gates, claims its Metamaterial
Microelectromechanical mirrors based LIDARS can scan 120 degrees
(MEMS) based LIDARS distribute horizontally and 25 degrees vertically.
the laser pulses via one or multiple [32]
tiny tilting mirrors, whose angle is
controlled by the voltage applied to
them. By substituting the mechanical
scanning hardware with an electro-
mechanical system, MEMS LIDARS can
achieve an accurate and power-ef-
ficient laser deflection, that is also
cost-efficient.[29]
LIDAR provides a 3D point cloud of the environment.
Image : Renishaw
24 25Interference from a source with the Among the three main active, ToF
same wavelength, or changes in based systems, SONAR is mainly used
reflectivity of surfaces due to wet- as a sensor for very close proximity
ness can affect the performance of due to the lower range of ultrasound
LIDAR sensors. LIDAR performance waves. RADAR cannot make out
can also be affected by external light, complex shapes, but it is able to see
including from other LIDARS.[33] While through adverse weather such as rain
traditional LIDAR sensors use 900 nm and fog. LIDAR can better sense an
wavelengths, new sensors are shifting object’s shape, but is shorter range
to 1500 nm enabling the vehicle to and more affected by ambient light
see objects 150-250 m away.[26,28] and weather conditions. Usually two
active sensor systems are used in
LIDAR has the benefits of having a conjunction, and if the aim is to only
relatively wide field of vision, with rely on one, LIDAR is often chosen.
potentially full 360 degree 3D cover- Secondly, active sensors are often
age (depending on the type of LIDAR used in conjunction with passive
chosen). Furthermore, it has a longer sensors (cameras).
range, more accurate distance esti-
mates compared to passive (optical)
sensors and lower computing cost.[17]
Its resolution however, is poorer and
laser safety can put limits on the laser
power used, which in turn can affect
the capabilities of the sensor.
These sensors have traditionally been
very expensive, with prices of tens of
thousands of dollars for the iconic
rooftop mounted 360 degree units.
However, prices are coming down: Long range RADAR Cameras LIDAR Short / Medium Ultrasound
Object detection, A combination of 3D environment mapping, range RADAR Close range object
Market leader Velodyne announced in
through rain, fog, dust. cameras for short-long object detection. Short-mid range detection. For objects
January 2020 a Metamaterials LIDAR range object detection.
Signal can bounce object detection. entering your lane.
that should ship for $100, albeit offer- around/underneath Broad spectrum of use Inc. side and rear For parking.
ing a narrower field of vision (60° vehiclesin front that
cases: from distant collision avoidance.
horizontal x 10° vertical) and shorter feature perception to
obstruct view.
cross traffic detection.
range (100m).[34,35] Road sign recognition.
Various object detection and mapping
sensors are used for various purposes,
and have complementary capabilities
and ranges. Image: Wevolver
26 27Choice of Sensors The following technical factors Vehicle manufacturers use a
affect the choice of sensors: mixture of optical and ToF sen-
While all the sensors presented have sors, with sensors strategically Comparison of various sensors used in autonomous vehicles. [14,18,26,36–38]
their own strengths and shortcom- • The scanning range, determining located to overcome the short-
ings, no single one would be a viable the amount of time you have to comings of the specific technol-
solution for all conditions on the react to an object that is being ogy. By looking at their setup we
road. A vehicle needs to be able to sensed. can see example combinations
avoid close objects, while also sens- • Resolution, determining how used for perception: Measurement Data rate
ing objects far away from it. It needs much detail the sensor can give Sensor Cost ($)
distance (m) (Mbps)
to be able to operate in different you. • Tesla’s Model S uses a forward
environmental and road conditions • Field of view or the angular res- mounted radar to sense the
with challenging light and weather olution, determining how many road, 3 forward facing cameras
Camera 0-250 4–200 500-3500
circumstances. This means that to sensors you would need to cover to identify road signs, lanes and
reliably and safely operate an auton- the area you want to perceive. objects, and 12 ultrasonic sensors
omous vehicle, usually a mixture of • Ability to distinguish between to detect nearby obstacles around
sensors is utilized. multiple static and moving ob- the car Ultrasound 0.02-10 30-400 < 0.01
jects in 3D, determining the num- • Volvo-Uber uses a top mounted
ber of objects you can track. 360 degree Lidar to detect road
• Refresh rate, determining how objects, short and long range
frequently the information from optical cameras to identify road RADAR 0.2-300 30-400 0.1-15
the sensor is updated. signals and radar to sense close-
• General reliability and accuracy by obstacles
in different environmental con- • Waymo uses a 360 degree LIDAR
ditions. to detect road objects, 9 visual LIDAR Up to 250 1,000-75,000 20-100
• Cost, size and software compat- cameras to track the road and a
ibility. radar for obstacle identification
• Amount of data generated. near the car.
• Wayve uses a row of 2.3-meg- Note that these are typical ranges and more extreme values exist. For example, Arbe Robotics’ RADAR can
apixel RGB cameras with high-dy- generate 1GBps depending on requirements from OEMs. Also note that multiple low costs sensors can be
namic range, and satellite naviga- required to achieve comparable performance to high-end sensors.
tion to drive autonomously.[39]
28 29Different Approaches 3x Forward Facing Cameras (Wide, Main, Narrow) Forward Looking Side Cameras Rear View Camera
by Tesla, Volvo-Uber, and Waymo:
Tesla Model S. Volvo-Uber XC90.Way- Volvo provides a base vehicle with
mo Chrysler Pacifica[36, 40-45] Images: pre-wiring and harnessing for Uber
adapted from Tesla, Volvo, Waymo, by to directly plug in its own self-driv-
Wevolver. ing hardware, which includes the rig
with LIDAR and cameras on top of the
Companies take different approaches vehicle.
to the set of sensors used for autono-
my, and where they are placed around
the vehicle.
Tesla’s sensors contain heating to
counter frost and fog, Volvo’s camer-
as come equipped with a water-jet
washing system for cleaning their
nozzles, and the cone that contains
the cameras on Waymo’s Chrysler has
water jets and wipers for cleaning.
Forward Facing RADAR Rearward Looking Side Cameras 12 Ultrasonics around the vehicle
Uber’s Hardware:
Forward Facing Cameras LIDAR Side and Rear Cameras
4x RADAR Long-range LIDAR 360° Cameras Audio 2x Short-range LIDAR 2x Mid-range LIDAR
Volvo’s Hardware:
RADAR, front & back Forward Facing Cameras Side Cameras Ultrasound, front & back Rear Camera
30 31Geolocalization accuracy can be achieved using mul-
ti-constellation; where the receiver
In the absence of additional signals
or onboard sensors, dead-reckoning
Maps
leverages signals from multiple GNSS may be used, where the car’s naviga- Today, map services such as Google
Once the autonomous vehicle has systems. Furthermore, accuracy can be tion system uses wheel circumference, Maps are widely used for navigation.
scanned its environment, it can find brought down to ~ 1cm levels using speed, and steering direction data to However, autonomous vehicles will
its location on the road relative to additional technologies that augment calculate a position from occasion- likely need a new class of high defi-
other objects around it. This informa- the GNSS system. ally received GPS data and the last nition (HD) maps that represent the
tion is critical for lower-level path known position.[52] In a smart city world at up to two orders of magni-
planning to avoid any collisions with To identify the position of the car, all environment, additional navigational tude more detail. With an accuracy of
objects in the vehicle’s immediate satellite navigation systems rely on aid can be provided by transponders a decimeter or less, HD maps increase
vicinity. the time of flight of a signal between that provide a signal to the car; by the spatial and contextual awareness
the receiver and a set of satellites. measuring its distance from two or “If we want to have of autonomous vehicles and provide
On top of that, in most cases the user GNSS receivers triangulate their po- more signals the vehicle can find its a source of redundancy for their
communicates the place they would sition using their calculated distance location within the environment.
autonomous cars sensors.
like to go to in terms of a geograph- from at least four satellites.[48] By con- everywhere, we have
ical location, which translates to a tinuously sensing, the path of the ve- to have digital maps By triangulating the distance from
latitude and longitude. Hence, in addi- hicle is revealed. The heading of the known objects in a HD map, the
tion to knowing its relative position vehicle can be determined using two everywhere.” precise localization of a vehicle can
in the local environment, the vehicle GNSS antennas, by using dedicated
needs to know its global position on onboard sensors such as a compass, Amnon Shashua,
Chief Technology Officer at
Earth in order to be able to determine or it can be calculated based on input Mobileye, 2017 [55]
a path towards the user’s destination. from vision sensors.[49]
The default geolocalization method While accurate, GNSS systems are
is satellite navigation, which provides also affected by environmental fac-
a general reference frame for where tors such as cloud cover and signal
the vehicle is located on the planet. reflection. In addition, signals can be
Different Global Navigation Satellite blocked by man-made objects such as
Systems (GNSS) such as the American tunnels or large structures. In some
GPS, the Russian GLONASS, the Euro- countries or regions, the signal might
pean Galileo or the Chinese Beidou also be too weak to accurately geolo-
can provide positioning information cate the vehicle.
with horizontal and vertical resolu-
tions of a few meters. To avoid geolocalization issues, an
Inertial Measurement Unit (IMU) is
While GPS guarantees a global signal integrated with the system.[50,51] By
user range error (URE) of less than 7.8 using gyroscopes and accelerometers,
m, its signal’s actual average range such a unit can extrapolate the data
error has been less than 0.71 m. The available to estimate the new loca-
real accuracy for a user however, de- tion of the vehicle when GNSS data is
pends on local factors such as signal unavailable.
blockage, atmospheric conditions, and
quality of the receiver that’s used.
[46]
Galileo, once fully operational,
could deliver a < 1m URE.[47] Higher
A 3D HD map covering an intersection. Image: Here
32 33be determined. Another benefit is
that the detailed information a high
definition map contains could narrow
As another example, London based
startup Wayve only uses standard
sat-nav and cameras. They aim to
Thinking & Learning
down the information that a vehicle’s achieve full autonomy by using
perception system needs to acquire, imitation learning algorithms to copy
and enable the sensors and software the behavior of expert human drivers, Based on the raw data captured SLAM and In order to perform SLAM more accu-
to dedicate more efforts towards and consequently using reinforcement by the AV’s sensor suite and the rately, sensor fusion comes into play.
moving objects.[53] learning to learn from each inter- pre-existing maps it has access to, Sensor Fusion Sensor fusion is the process of com-
vention of their human safety driver the automated driving system needs bining data from multiple sensors
HD maps can represent lanes, geome- while training the model in autono- to construct and update a map of SLAM is a complex process because and databases to achieve improved
try, traffic signs, the road surface, and mous mode.[58] its environment while keeping track a map is needed for localization and information. It is a multi-level pro-
the location of objects like trees. The of its location in it. Simultaneous a good position estimate is needed cess that deals with the association,
information in such a map is repre- Researchers from MIT’s Computer localization and mapping (SLAM) al- for mapping. Though long consid- correlation, and combination of data,
sented in layers, with generally at Science and Artificial Intelligence gorithms let the vehicle achieve just ered a fundamental chicken-or-egg and enables to achieve less expen-
least one of the layers containing 3D Laboratory (CSAIL) also took a that. Once its location on its map is problem for robots to become au- sive, higher quality, or more relevant
geometric information of the world in ‘map-less’ approach and developed a known, the system can start planning tonomous, breakthrough research in information than when using a single
high detail to enable precise calcu- system that uses LIDAR sensors for which path to take to get from one the mid-1980s and 90s solved SLAM data source alone.[64]
lations. all aspects of navigation, only relying point to another. on a conceptual and theoretical
on GPS for a rough location estimate. level. Since then, a variety of SLAM
Challenges lie in the large efforts to [59–61]
approaches have been developed, the
generate high definition maps and majority of which uses probabilistic
keep them up to date, as well as in concepts.[62,63]
the large amount of data storage and
bandwidth it takes to store and trans-
fer these maps.[54]
Most in the industry express HD maps
to be a necessity for high levels of
autonomy, in any case for the near SENSING & DATA INPUT COMPUTATION & DECISION MAKING ACT & CONTROL
future as they have to make up for THE VEHICLE
limited abilities of AI. However, some
Cameras (inc. Thermal Cameras)
disagree or take a different approach.
RADAR
According to Elon musk Tesla “briefly
barked up the tree of high precision LIDAR Steering
lane line [maps], but decided it wasn’t
Ultrasound Sensors Simultaneous Accelerating
a good idea.”[56] In 2015 Apple, for Localization Planning
its part, patented an autonomous IMU
And
Braking
Mapping
navigation system that lets a vehicle
navigate without referring to exter- GNSS Signalling
nal data sources. The system in the
patent leverages AI capabilities and “The need for dense Map Data
vehicle sensors instead.[57]
3-D maps limits Vehicle-to-Vehicle Communication
the places where Vehicle-to-Infrastructure Communication
self-driving cars can
operate.”
Daniela Rus, The complex computation and
director of MIT’s Computer decision making environment of
Science and Artificial Intelli- an autonomous vehicle.[65]
gence Laboratory (CSAIL), 2018 Image: Wevolver
34 35For the all processing and decision The question which approach is best First, we’ll review how the data from es the transformation between the
making required to go from sensor for AVs is an area of ongoing debate. the sensors is processed to reach a two point clouds, which enables to
data to motion in general two differ- The traditional, and most common decision regarding the robotic vehi- calculate the translation and rotation
ent AI approaches are used [66]: approach consists of decomposing cle’s motion. Depending on the differ- the vehicle had.
the problem of autonomous driv- ent sensors used onboard the vehicle,
1. Sequentially, where the driving ing into a number of sub-problems different software schemes can be While useful, the preceding ap-
process is decomposed into com- and solving each one sequentially used to extract useful information proaches consume much computing
ponents of a hierarchical pipeline. with a dedicated machine learning from the sensor signals. time, and cannot easily be scaled
Each step (sensing, localization algorithm technique from computer to the case of a self-driving vehicle
and mapping, path planning, vision, sensor fusion, localization, There are several algorithms that operating in a continuously changing
motion control) is handled by a control theory, and path planning.[67] can be used to identify objects in environment. That is where machine
specific software element, with an image. The simplest approach learning comes into play, relying on
each component of the pipeline End-to-End (e2e) learning increas- is edge detection, where changes computer algorithms that have al-
feeding data to the next one, or ingly gets interest as a potential in the intensity of light or color in ready learned to perform a task from
2. An End-to-End solution based on solution to the challenges of the different pixels are assessed.[69] One existing data.
deep learning that takes care of complex AI systems for autonomous would expect pixels that belong to
all these functions. vehicles. End-to-end (e2e) learning the same object to have similar light
applies iterative learning to a com- properties; hence looking at chang-
plex system as a whole, and has been es in the light intensity can help
popularized in the context of deep separate objects or detect where one
learning. An End-to-End approach object turns to the next. The problem
attempts to create an autonomous with this approach is that in low light
driving system with a single, com- intensity (say at night) the algorithm
prehensive software component that cannot perform well since it relies on
directly maps sensor inputs to driving differences in light intensity. In addi-
actions. Because of breakthroughs tion, as this analysis has to be done
in deep learning the capabilities of on each shot and on multiple pixels,
e2e systems have increased as such there is a high computational cost.
that they are now considered a viable
option. These systems can be created LIDAR data can be used to compute
Perception & High-Level Behavior Motion Controllers Autonomy with one or multiple different types the movement of the vehicle with
+ + + =
Localization Path Planning Arbitration of machine learning methods, such the same principle. By comparing
(low-level path as Convolutional Neural Networks or two point clouds taken at consecu-
planning) Reinforcement Learning, which we tive instants, some objects will have
will elaborate on later in this report. moved closer or further from the
[67,68]
sensor. A software technique called
iterative closest point iteratively revis-
//
End2End Learning = Autonomy
Two main approaches to the AI architecture of autonomous vehicles: 1) sequential per-
ception-planning-action-pipelines 2) an End2End system.[66]
Image: Wevolver
36 37Machine Learning CNNs are mainly used to process RNNs are powerful tools when work- These methods don’t necessarily sit in
Methods images and spatial information to
extract features of interest and identi-
ing with temporal information such
as videos. In these networks the out-
isolation. For example, companies like
Tesla rely on hybrid forms, which try
Different types of machine learning fy objects in the environment. These puts from the previous steps are fed to use multiple methods together to
algorithms are currently being used neural networks are made of a convo- into the network as input, allowing increase accuracy and reduce compu-
for different applications in autono- lution layer: a collection of filters that information and knowledge to persist tational demands.[77,78]
mous vehicles. In essence, machine tries to distinguish elements of an im- in the network and be contextualized.
learning maps a set of inputs to a set age or input data to label them. The [72–74]
Training networks on several tasks
of outputs, based on a set of training output of this convolution layer is fed at once is a common practice in
data provided. Convolutional Neural into an algorithm that combines them DRL combines Deep Learning (DL) deep learning, often called multi-task
Networks (CNN), Recurrent Neural to predict the best description of an and Reinforcement Learning. DRL training or auxiliary task training. This
Networks (RNN) and Deep Reinforce- image. The final software component methods let software-defined ‘agents’ is to avoid overfitting, a common
ment Learning (DRL) are the most is usually called an object classifier, learn the best possible actions to issue with neural networks. When a
common deep learning methodolo- as it can categorize an object in the achieve their goals in a virtual en- machine learning algorithm is trained
gies applied to autonomous driving. image, for example a street sign or vironment using a reward function. for a particular task, it can become
[66]
another car.[69–71] These goal-oriented algorithms learn so focused imitating the data it is
how to attain an objective, or how to trained on that its output becomes
maximize along a specific dimension unrealistic when an interpolation or
over many steps. While promising, a extrapolation is attempted. By train-
challenge for DRL is the design of the ing the machine learning algorithm
correct reward function for driving a on multiple tasks, the core of the
vehicle. Deep Reinforcement Learning network will specialize in finding
is considered to be still in an early general features that are useful for
stage regarding application in auton- all purposes instead of specializing
omous vehicles.[75,76] only on one task. This can make the
outputs more realistic and useful for
applications.
Algorithms turn input from sensors into object classifications and a map of the environment.
Image: Wayve
38 39Gathering Data One way to gather data is by using a
prototype car. These cars are driven
In order for these algorithms to be by a driver. The perception sensors
used, they need to be trained on data onboard are used to gather informa-
sets that represent realistic scenarios. tion about the environment. At the
With any machine learning process, a same time, an on-board computer will
part of the data set is used for train- record sensors readings coming from
ing, and another part for validation the pedals, the steering wheel, and all
and testing. As such, a great amount other information that can describe
of data is annotated by autonomous how the driver acts. Due to the large
vehicle companies to achieve this amount of data that needs to be
goal.[77] Many datasets, with semantic gathered and labelled by humans,
segmentation of street objects, sign this is a costly process. According
classification, pedestrian detection to Andrej Karpathy, Director of AI at
and depth prediction, have been Tesla, most of the efforts in his group
made openly available by researchers are dedicated to getting better and
and companies including Aptiv, Lyft, better data.[77]
Waymo, and Baidu. This has signifi-
cantly helped to push the capabilities Alternatively, simulators may be used.
of the machine learning algorithms “Current physical testing isn’t enough;
forward.[79–81] therefore, virtual testing will be
required,” says Jamie Smith, Director
of Global Automotive Strategy at
National Instruments.[82] By building
realistic simulators, software compa-
nies can create thousands of virtual
scenarios. This brings the cost of data
acquisition down but introduces the
problem of realism: these virtual
scenarios are defined by humans and
are less random that what a real vehi-
cle goes through. There is growing
research in this area, called sim-to-
real transfer, that studies methods to
transfer the knowledge gathered in
simulation in the real world.[83]
Using all the data from the sensors
and these algorithms, an autonomous
vehicle can detect objects surround-
ing it. Next, it needs to find a path to
“We have quite a follow.
good simulation, too, “At Waymo, we’ve
but it just does not driven more than 10
capture the long tail million miles in the
of weird things that real world, and over
happen in the real 10 billion miles in
world.” simulation.”
Elon Musk, Waymo CTO Dmitri Dolgov, Simulators are used to explore thousands of varia-
April 2019 [84] July 2019 [85] ble scenarios. Image: Autoware.AI
40 41Path Planning Training neural networks and infer- “In most cases, if you look at what went wrong
ence during operations of the vehicle
requires enormous computing power.
during a disengagement [the moment when
With the vehicle knowing the objects Until recently, most machine learning the AV needs human intervention - note by
in its environment and its location, tasks were executed on cloud-based
the large scale path of the vehicle can infrastructure with excessive comput- editor], the role of hardware failure is 0.0 per-
be determined by using a voronoi di- ing power and cooling. With autono- cent. Most of the time, it’s a software failure,
agram (maximizing distance between mous vehicles, that is no longer possi-
vehicle and objects), an occupancy ble as the vehicle needs to be able to that is, software failing to predict what the
grid algorithm, or with a driving corri- simultaneously react to new data. As vehicles are gonna be doing or what the pe-
dors algorithm.[86] However, these tra- such, part of the processing required
ditional approaches are not enough to operate the vehicle needs to take destrians are gonna be doing.”
for a vehicle that is interacting with place onboard, while model refine-
other moving objects around it and ments could be done on the cloud.
their output needs to be fine-tuned. Anthony Levandowski,
Recent advances in machine learning autonomous vehicle technology
Some autonomous vehicles rely on are focusing on how the huge amount
pioneer, April 2019 [90]
machine learning algorithms to not of data generated by the sensors on-
only perceive their environment but board AVs can be efficiently processed
also to act on that data to control to reduce the computational cost,
the car. Path planning can be taught using concepts such as attention [88]
to a CNN through imitation learning, or core-sets.[89] In addition, advances
in which the CNN tries to imitate the in chip manufacturing and miniatur-
behavior of a driver. In more advanced ization are increasing the computing
algorithms, DRL is used, where a capacity that can be mounted on an
reward is provided to the autonomous autonomous vehicle. With advances
system for driving in an acceptable in networking protocols, cars might
manner. Usually, these methods be able to rely on low-latency net-
are hybridized with more classical work-based processing of data to aid
methods of motion planning and them in their autonomous operation.
trajectory optimization to make sure
that the paths are robust. In addition,
manufacturers can include additional
objectives, such as reducing fuel use,
for the model to take into account as
it tries to identify optimal paths.[87]
Autonomous vehicles deploy algorithms to plan the vehi-
cle’s own path, as well as estimate the path of other moving
objects (in this case the system also estimates the path of
the 2 red squares that represent bicyclists). Image: Waymo
42 43You can also read
