Towards Building Self-Aware Object Detectors via Reliable Uncertainty
  Quantification and Calibration
Kemal Oksuz Tom Joy Puneet K. Dokania
The current approach for testing the robustness of object detectors suffers
from serious deficiencies such as improper methods of performing
out-of-distribution detection and using calibration metrics which do not
consider both localisation and classification quality. In this work, we address
these issues, and introduce the Self-Aware Object Detection (SAOD) task, a
unified testing framework which respects and adheres to the challenges that
object detectors face in safety-critical environments such as autonomous
driving. Specifically, the SAOD task requires an object detector to be: robust
to domain shift; obtain reliable uncertainty estimates for the entire scene;
and provide calibrated confidence scores for the detections. We extensively use
our framework, which introduces novel metrics and large scale test datasets, to
test numerous object detectors in two different use-cases, allowing us to
highlight critical insights into their robustness performance. Finally, we
introduce a simple baseline for the SAOD task, enabling researchers to
benchmark future proposed methods and move towards robust object detectors
which are fit for purpose. Code is available at https://github.com/fiveai/saod