Selective Prediction for Semantic Segmentation using Post-Hoc Confidence
  Estimation and Its Performance under Distribution Shift
Bruno Laboissiere Camargos Borges Bruno Machado Pacheco Danilo Silva
Semantic segmentation plays a crucial role in various computer vision
applications, yet its efficacy is often hindered by the lack of high-quality
labeled data. To address this challenge, a common strategy is to leverage
models trained on data from different populations, such as publicly available
datasets. This approach, however, leads to the distribution shift problem,
presenting a reduced performance on the population of interest. In scenarios
where model errors can have significant consequences, selective prediction
methods offer a means to mitigate risks and reduce reliance on expert
supervision. This paper investigates selective prediction for semantic
segmentation in low-resource settings, thus focusing on post-hoc confidence
estimators applied to pre-trained models operating under distribution shift. We
propose a novel image-level confidence measure tailored for semantic
segmentation and demonstrate its effectiveness through experiments on three
medical imaging tasks. Our findings show that post-hoc confidence estimators
offer a cost-effective approach to reducing the impacts of distribution shift.