Does Physical Adversarial Example Really Matter to Autonomous Driving?
  Towards System-Level Effect of Adversarial Object Evasion Attack
Ningfei Wang Yunpeng Luo Takami Sato Kaidi Xu Qi Alfred Chen
In autonomous driving (AD), accurate perception is indispensable to achieving
safe and secure driving. Due to its safety-criticality, the security of AD
perception has been widely studied. Among different attacks on AD perception,
the physical adversarial object evasion attacks are especially severe. However,
we find that all existing literature only evaluates their attack effect at the
targeted AI component level but not at the system level, i.e., with the entire
system semantics and context such as the full AD pipeline. Thereby, this raises
a critical research question: can these existing researches effectively achieve
system-level attack effects (e.g., traffic rule violations) in the real-world
AD context? In this work, we conduct the first measurement study on whether and
how effectively the existing designs can lead to system-level effects,
especially for the STOP sign-evasion attacks due to their popularity and
severity. Our evaluation results show that all the representative prior works
cannot achieve any system-level effects. We observe two design limitations in
the prior works: 1) physical model-inconsistent object size distribution in
pixel sampling and 2) lack of vehicle plant model and AD system model
consideration. Then, we propose SysAdv, a novel system-driven attack design in
the AD context and our evaluation results show that the system-level effects
can be significantly improved, i.e., the violation rate increases by around
70%.