SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors
Hongge Chen Zhao Chen Gregory P. Meyer Dennis Park Carl Vondrick Ashish Shrivastava Yuning Chai
We present SHIFT3D, a differentiable pipeline for generating 3D shapes that
are structurally plausible yet challenging to 3D object detectors. In
safety-critical applications like autonomous driving, discovering such novel
challenging objects can offer insight into unknown vulnerabilities of 3D
detectors. By representing objects with a signed distanced function (SDF), we
show that gradient error signals allow us to smoothly deform the shape or pose
of a 3D object in order to confuse a downstream 3D detector. Importantly, the
objects generated by SHIFT3D physically differ from the baseline object yet
retain a semantically recognizable shape. Our approach provides interpretable
failure modes for modern 3D object detectors, and can aid in preemptive
discovery of potential safety risks within 3D perception systems before these
risks become critical failures.