Calibration-based Dual Prototypical Contrastive Learning Approach for
  Domain Generalization Semantic Segmentation
Muxin Liao Shishun Tian Yuhang Zhang Guoguang Hua Wenbin Zou Xia Li
Prototypical contrastive learning (PCL) has been widely used to learn
class-wise domain-invariant features recently. These methods are based on the
assumption that the prototypes, which are represented as the central value of
the same class in a certain domain, are domain-invariant. Since the prototypes
of different domains have discrepancies as well, the class-wise
domain-invariant features learned from the source domain by PCL need to be
aligned with the prototypes of other domains simultaneously. However, the
prototypes of the same class in different domains may be different while the
prototypes of different classes may be similar, which may affect the learning
of class-wise domain-invariant features. Based on these observations, a
calibration-based dual prototypical contrastive learning (CDPCL) approach is
proposed to reduce the domain discrepancy between the learned class-wise
features and the prototypes of different domains for domain generalization
semantic segmentation. It contains an uncertainty-guided PCL (UPCL) and a
hard-weighted PCL (HPCL). Since the domain discrepancies of the prototypes of
different classes may be different, we propose an uncertainty probability
matrix to represent the domain discrepancies of the prototypes of all the
classes. The UPCL estimates the uncertainty probability matrix to calibrate the
weights of the prototypes during the PCL. Moreover, considering that the
prototypes of different classes may be similar in some circumstances, which
means these prototypes are hard-aligned, the HPCL is proposed to generate a
hard-weighted matrix to calibrate the weights of the hard-aligned prototypes
during the PCL. Extensive experiments demonstrate that our approach achieves
superior performance over current approaches on domain generalization semantic
segmentation tasks.