Average Calibration Error: A Differentiable Loss for Improved
  Reliability in Image Segmentation
Theodore Barfoot Luis Garcia-Peraza-Herrera Ben Glocker Tom Vercauteren
Deep neural networks for medical image segmentation often produce
overconfident results misaligned with empirical observations. Such
miscalibration, challenges their clinical translation. We propose to use
marginal L1 average calibration error (mL1-ACE) as a novel auxiliary loss
function to improve pixel-wise calibration without compromising segmentation
quality. We show that this loss, despite using hard binning, is directly
differentiable, bypassing the need for approximate but differentiable surrogate
or soft binning approaches. Our work also introduces the concept of dataset
reliability histograms which generalises standard reliability diagrams for
refined visual assessment of calibration in semantic segmentation aggregated at
the dataset level. Using mL1-ACE, we reduce average and maximum calibration
error by 45% and 55% respectively, maintaining a Dice score of 87% on the BraTS
2021 dataset. We share our code here: https://github.com/cai4cai/ACE-DLIRIS