Cal-DETR: Calibrated Detection Transformer
Muhammad Akhtar Munir Salman Khan Muhammad Haris Khan Mohsen Ali Fahad Shahbaz Khan
Albeit revealing impressive predictive performance for several computer
vision tasks, deep neural networks (DNNs) are prone to making overconfident
predictions. This limits the adoption and wider utilization of DNNs in many
safety-critical applications. There have been recent efforts toward calibrating
DNNs, however, almost all of them focus on the classification task.
Surprisingly, very little attention has been devoted to calibrating modern
DNN-based object detectors, especially detection transformers, which have
recently demonstrated promising detection performance and are influential in
many decision-making systems. In this work, we address the problem by proposing
a mechanism for calibrated detection transformers (Cal-DETR), particularly for
Deformable-DETR, UP-DETR and DINO. We pursue the train-time calibration route
and make the following contributions. First, we propose a simple yet effective
approach for quantifying uncertainty in transformer-based object detectors.
Second, we develop an uncertainty-guided logit modulation mechanism that
leverages the uncertainty to modulate the class logits. Third, we develop a
logit mixing approach that acts as a regularizer with detection-specific losses
and is also complementary to the uncertainty-guided logit modulation technique
to further improve the calibration performance. Lastly, we conduct extensive
experiments across three in-domain and four out-domain scenarios. Results
corroborate the effectiveness of Cal-DETR against the competing train-time
methods in calibrating both in-domain and out-domain detections while
maintaining or even improving the detection performance. Our codebase and
pre-trained models can be accessed at
\url{https://github.com/akhtarvision/cal-detr}.