Neighbor-Aware Calibration of Segmentation Networks with Penalty-Based
  Constraints
Balamurali Murugesan Sukesh Adiga Vasudeva Bingyuan Liu Hervé Lombaert Ismail Ben Ayed Jose Dolz
Ensuring reliable confidence scores from deep neural networks is of paramount
significance in critical decision-making systems, particularly in real-world
domains such as healthcare. Recent literature on calibrating deep segmentation
networks has resulted in substantial progress. Nevertheless, these approaches
are strongly inspired by the advancements in classification tasks, and thus
their uncertainty is usually modeled by leveraging the information of
individual pixels, disregarding the local structure of the object of interest.
Indeed, only the recent Spatially Varying Label Smoothing (SVLS) approach
considers pixel spatial relationships across classes, by softening the pixel
label assignments with a discrete spatial Gaussian kernel. In this work, we
first present a constrained optimization perspective of SVLS and demonstrate
that it enforces an implicit constraint on soft class proportions of
surrounding pixels. Furthermore, our analysis shows that SVLS lacks a mechanism
to balance the contribution of the constraint with the primary objective,
potentially hindering the optimization process. Based on these observations, we
propose NACL (Neighbor Aware CaLibration), a principled and simple solution
based on equality constraints on the logit values, which enables to control
explicitly both the enforced constraint and the weight of the penalty, offering
more flexibility. Comprehensive experiments on a wide variety of well-known
segmentation benchmarks demonstrate the superior calibration performance of the
proposed approach, without affecting its discriminative power. Furthermore,
ablation studies empirically show the model agnostic nature of our approach,
which can be used to train a wide span of deep segmentation networks.