DUSA: Decoupled Unsupervised Sim2Real Adaptation for
  Vehicle-to-Everything Collaborative Perception
Xianghao Kong Wentao Jiang Jinrang Jia Yifeng Shi Runsheng Xu Si Liu
Vehicle-to-Everything (V2X) collaborative perception is crucial for
autonomous driving. However, achieving high-precision V2X perception requires a
significant amount of annotated real-world data, which can always be expensive
and hard to acquire. Simulated data have raised much attention since they can
be massively produced at an extremely low cost. Nevertheless, the significant
domain gap between simulated and real-world data, including differences in
sensor type, reflectance patterns, and road surroundings, often leads to poor
performance of models trained on simulated data when evaluated on real-world
data. In addition, there remains a domain gap between real-world collaborative
agents, e.g. different types of sensors may be installed on autonomous vehicles
and roadside infrastructures with different extrinsics, further increasing the
difficulty of sim2real generalization. To take full advantage of simulated
data, we present a new unsupervised sim2real domain adaptation method for V2X
collaborative detection named Decoupled Unsupervised Sim2Real Adaptation
(DUSA). Our new method decouples the V2X collaborative sim2real domain
adaptation problem into two sub-problems: sim2real adaptation and inter-agent
adaptation. For sim2real adaptation, we design a Location-adaptive Sim2Real
Adapter (LSA) module to adaptively aggregate features from critical locations
of the feature map and align the features between simulated data and real-world
data via a sim/real discriminator on the aggregated global feature. For
inter-agent adaptation, we further devise a Confidence-aware Inter-agent
Adapter (CIA) module to align the fine-grained features from heterogeneous
agents under the guidance of agent-wise confidence maps. Experiments
demonstrate the effectiveness of the proposed DUSA approach on unsupervised
sim2real adaptation from the simulated V2XSet dataset to the real-world
DAIR-V2X-C dataset.