SegDA: Maximum Separable Segment Mask with Pseudo Labels for Domain
  Adaptive Semantic Segmentation
Anant Khandelwal
Unsupervised Domain Adaptation (UDA) aims to solve the problem of label
scarcity of the target domain by transferring the knowledge from the label rich
source domain. Usually, the source domain consists of synthetic images for
which the annotation is easily obtained using the well known computer graphics
techniques. However, obtaining annotation for real world images (target domain)
require lot of manual annotation effort and is very time consuming because it
requires per pixel annotation. To address this problem we propose SegDA module
to enhance transfer performance of UDA methods by learning the maximum
separable segment representation. This resolves the problem of identifying
visually similar classes like pedestrian/rider, sidewalk/road etc. We leveraged
Equiangular Tight Frame (ETF) classifier inspired from Neural Collapse for
maximal separation between segment classes. This causes the source domain pixel
representation to collapse to a single vector forming a simplex vertices which
are aligned to the maximal separable ETF classifier. We use this phenomenon to
propose the novel architecture for domain adaptation of segment representation
for target domain. Additionally, we proposed to estimate the noise in labelling
the target domain images and update the decoder for noise correction which
encourages the discovery of pixels for classes not identified in pseudo labels.
We have used four UDA benchmarks simulating synthetic-to-real,
daytime-to-nighttime, clear-to-adverse weather scenarios. Our proposed approach
outperforms +2.2 mIoU on GTA -> Cityscapes, +2.0 mIoU on Synthia -> Cityscapes,
+5.9 mIoU on Cityscapes -> DarkZurich, +2.6 mIoU on Cityscapes -> ACDC.