Beyond Classification: Definition and Density-based Estimation of
  Calibration in Object Detection
Teodora Popordanoska Aleksei Tiulpin Matthew B. Blaschko
Despite their impressive predictive performance in various computer vision
tasks, deep neural networks (DNNs) tend to make overly confident predictions,
which hinders their widespread use in safety-critical applications. While there
have been recent attempts to calibrate DNNs, most of these efforts have
primarily been focused on classification tasks, thus neglecting DNN-based
object detectors. Although several recent works addressed calibration for
object detection and proposed differentiable penalties, none of them are
consistent estimators of established concepts in calibration. In this work, we
tackle the challenge of defining and estimating calibration error specifically
for this task. In particular, we adapt the definition of classification
calibration error to handle the nuances associated with object detection, and
predictions in structured output spaces more generally. Furthermore, we propose
a consistent and differentiable estimator of the detection calibration error,
utilizing kernel density estimation. Our experiments demonstrate the
effectiveness of our estimator against competing train-time and post-hoc
calibration methods, while maintaining similar detection performance.