Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with
  Differentiable Expected Calibration Error
Zixin Wang Yadan Luo Zhi Chen Sen Wang Zi Huang
The prevalence of domain adaptive semantic segmentation has prompted concerns
regarding source domain data leakage, where private information from the source
domain could inadvertently be exposed in the target domain. To circumvent the
requirement for source data, source-free domain adaptation has emerged as a
viable solution that leverages self-training methods to pseudo-label
high-confidence regions and adapt the model to the target data. However, the
confidence scores obtained are often highly biased due to over-confidence and
class-imbalance issues, which render both model selection and optimization
problematic. In this paper, we propose a novel calibration-guided source-free
domain adaptive semantic segmentation (Cal-SFDA) framework. The core idea is to
estimate the expected calibration error (ECE) from the segmentation
predictions, serving as a strong indicator of the model's generalization
capability to the unlabeled target domain. The estimated ECE scores, in turn,
assist the model training and fair selection in both source training and target
adaptation stages. During model pre-training on the source domain, we ensure
the differentiability of the ECE objective by leveraging the LogSumExp trick
and using ECE scores to select the best source checkpoints for adaptation. To
enable ECE estimation on the target domain without requiring labels, we train a
value net for ECE estimation and apply statistic warm-up on its BatchNorm
layers for stability. The estimated ECE scores assist in determining the
reliability of prediction and enable class-balanced pseudo-labeling by
positively guiding the adaptation progress and inhibiting potential error
accumulation. Extensive experiments on two widely-used synthetic-to-real
transfer tasks show that the proposed approach surpasses previous
state-of-the-art by up to 5.25% of mIoU with fair model selection criteria.