Mx2M: Masked Cross-Modality Modeling in Domain Adaptation for 3D
  Semantic Segmentation
Boxiang Zhang Zunran Wang Yonggen Ling Yuanyuan Guan Shenghao Zhang Wenhui Li
Existing methods of cross-modal domain adaptation for 3D semantic
segmentation predict results only via 2D-3D complementarity that is obtained by
cross-modal feature matching. However, as lacking supervision in the target
domain, the complementarity is not always reliable. The results are not ideal
when the domain gap is large. To solve the problem of lacking supervision, we
introduce masked modeling into this task and propose a method Mx2M, which
utilizes masked cross-modality modeling to reduce the large domain gap. Our
Mx2M contains two components. One is the core solution, cross-modal removal and
prediction (xMRP), which makes the Mx2M adapt to various scenarios and provides
cross-modal self-supervision. The other is a new way of cross-modal feature
matching, the dynamic cross-modal filter (DxMF) that ensures the whole method
dynamically uses more suitable 2D-3D complementarity. Evaluation of the Mx2M on
three DA scenarios, including Day/Night, USA/Singapore, and A2D2/SemanticKITTI,
brings large improvements over previous methods on many metrics.