Identification of Fine-grained Systematic Errors via Controlled Scene
  Generation
Valentyn Boreiko Matthias Hein Jan Hendrik Metzen
Many safety-critical applications, especially in autonomous driving, require
reliable object detectors. They can be very effectively assisted by a method to
search for and identify potential failures and systematic errors before these
detectors are deployed. Systematic errors are characterized by combinations of
attributes such as object location, scale, orientation, and color, as well as
the composition of their respective backgrounds. To identify them, one must
rely on something other than real images from a test set because they do not
account for very rare but possible combinations of attributes. To overcome this
limitation, we propose a pipeline for generating realistic synthetic scenes
with fine-grained control, allowing the creation of complex scenes with
multiple objects. Our approach, BEV2EGO, allows for a realistic generation of
the complete scene with road-contingent control that maps 2D bird's-eye view
(BEV) scene configurations to a first-person view (EGO). In addition, we
propose a benchmark for controlled scene generation to select the most
appropriate generative outpainting model for BEV2EGO. We further use it to
perform a systematic analysis of multiple state-of-the-art object detection
models and discover differences between them.