MoPA: Multi-Modal Prior Aided Domain Adaptation for 3D Semantic
  Segmentation
Haozhi Cao Yuecong Xu Jianfei Yang Pengyu Yin Shenghai Yuan Lihua Xie
Multi-modal unsupervised domain adaptation (MM-UDA) for 3D semantic
segmentation is a practical solution to embed semantic understanding in
autonomous systems without expensive point-wise annotations. While previous
MM-UDA methods can achieve overall improvement, they suffer from significant
class-imbalanced performance, restricting their adoption in real applications.
This imbalanced performance is mainly caused by: 1) self-training with
imbalanced data and 2) the lack of pixel-wise 2D supervision signals. In this
work, we propose Multi-modal Prior Aided (MoPA) domain adaptation to improve
the performance of rare objects. Specifically, we develop Valid Ground-based
Insertion (VGI) to rectify the imbalance supervision signals by inserting prior
rare objects collected from the wild while avoiding introducing artificial
artifacts that lead to trivial solutions. Meanwhile, our SAM consistency loss
leverages the 2D prior semantic masks from SAM as pixel-wise supervision
signals to encourage consistent predictions for each object in the semantic
mask. The knowledge learned from modal-specific prior is then shared across
modalities to achieve better rare object segmentation. Extensive experiments
show that our method achieves state-of-the-art performance on the challenging
MM-UDA benchmark. Code will be available at https://github.com/AronCao49/MoPA.