Attacking Motion Planners Using Adversarial Perception Errors
Jonathan Sadeghi Nicholas A. Lord John Redford Romain Mueller
Autonomous driving (AD) systems are often built and tested in a modular
fashion, where the performance of different modules is measured using
task-specific metrics. These metrics should be chosen so as to capture the
downstream impact of each module and the performance of the system as a whole.
For example, high perception quality should enable prediction and planning to
be performed safely. Even though this is true in general, we show here that it
is possible to construct planner inputs that score very highly on various
perception quality metrics but still lead to planning failures. In an analogy
to adversarial attacks on image classifiers, we call such inputs
\textbf{adversarial perception errors} and show they can be systematically
constructed using a simple boundary-attack algorithm. We demonstrate the
effectiveness of this algorithm by finding attacks for two different black-box
planners in several urban and highway driving scenarios using the CARLA
simulator. Finally, we analyse the properties of these attacks and show that
they are isolated in the input space of the planner, and discuss their
implications for AD system deployment and testing.