CMDA: Cross-Modality Domain Adaptation for Nighttime Semantic
  Segmentation
Ruihao Xia Chaoqiang Zhao Meng Zheng Ziyan Wu Qiyu Sun Yang Tang
Most nighttime semantic segmentation studies are based on domain adaptation
approaches and image input. However, limited by the low dynamic range of
conventional cameras, images fail to capture structural details and boundary
information in low-light conditions. Event cameras, as a new form of vision
sensors, are complementary to conventional cameras with their high dynamic
range. To this end, we propose a novel unsupervised Cross-Modality Domain
Adaptation (CMDA) framework to leverage multi-modality (Images and Events)
information for nighttime semantic segmentation, with only labels on daytime
images. In CMDA, we design the Image Motion-Extractor to extract motion
information and the Image Content-Extractor to extract content information from
images, in order to bridge the gap between different modalities (Images to
Events) and domains (Day to Night). Besides, we introduce the first image-event
nighttime semantic segmentation dataset. Extensive experiments on both the
public image dataset and the proposed image-event dataset demonstrate the
effectiveness of our proposed approach. We open-source our code, models, and
dataset at https://github.com/XiaRho/CMDA.