Semantic Connectivity-Driven Pseudo-labeling for Cross-domain
  Segmentation
Dong Zhao Ruizhi Yang Shuang Wang Qi Zang Yang Hu Licheng Jiao Nicu Sebe Zhun Zhong
Presently, self-training stands as a prevailing approach in cross-domain
semantic segmentation, enhancing model efficacy by training with pixels
assigned with reliable pseudo-labels. However, we find two critical limitations
in this paradigm. (1) The majority of reliable pixels exhibit a speckle-shaped
pattern and are primarily located in the central semantic region. This presents
challenges for the model in accurately learning semantics. (2) Category noise
in speckle pixels is difficult to locate and correct, leading to error
accumulation in self-training. To address these limitations, we propose a novel
approach called Semantic Connectivity-driven pseudo-labeling (SeCo). This
approach formulates pseudo-labels at the connectivity level and thus can
facilitate learning structured and low-noise semantics. Specifically, SeCo
comprises two key components: Pixel Semantic Aggregation (PSA) and Semantic
Connectivity Correction (SCC). Initially, PSA divides semantics into 'stuff'
and 'things' categories and aggregates speckled pseudo-labels into semantic
connectivity through efficient interaction with the Segment Anything Model
(SAM). This enables us not only to obtain accurate boundaries but also
simplifies noise localization. Subsequently, SCC introduces a simple
connectivity classification task, which enables locating and correcting
connectivity noise with the guidance of loss distribution. Extensive
experiments demonstrate that SeCo can be flexibly applied to various
cross-domain semantic segmentation tasks, including traditional unsupervised,
source-free, and black-box domain adaptation, significantly improving the
performance of existing state-of-the-art methods. The code is available at
https://github.com/DZhaoXd/SeCo.