CoBEVFusion: Cooperative Perception with LiDAR-Camera Bird's-Eye View
  Fusion
Donghao Qiao Farhana Zulkernine
Autonomous Vehicles (AVs) use multiple sensors to gather information about
their surroundings. By sharing sensor data between Connected Autonomous
Vehicles (CAVs), the safety and reliability of these vehicles can be improved
through a concept known as cooperative perception. However, recent approaches
in cooperative perception only share single sensor information such as cameras
or LiDAR. In this research, we explore the fusion of multiple sensor data
sources and present a framework, called CoBEVFusion, that fuses LiDAR and
camera data to create a Bird's-Eye View (BEV) representation. The CAVs process
the multi-modal data locally and utilize a Dual Window-based Cross-Attention
(DWCA) module to fuse the LiDAR and camera features into a unified BEV
representation. The fused BEV feature maps are shared among the CAVs, and a 3D
Convolutional Neural Network is applied to aggregate the features from the
CAVs. Our CoBEVFusion framework was evaluated on the cooperative perception
dataset OPV2V for two perception tasks: BEV semantic segmentation and 3D object
detection. The results show that our DWCA LiDAR-camera fusion model outperforms
perception models with single-modal data and state-of-the-art BEV fusion
models. Our overall cooperative perception architecture, CoBEVFusion, also
achieves comparable performance with other cooperative perception models.