X-Align++: cross-modal cross-view alignment for Bird's-eye-view
  segmentation
Shubhankar Borse Senthil Yogamani Marvin Klingner Varun Ravi Hong Cai Abdulaziz Almuzairee Fatih Porikli
Bird's-eye-view (BEV) grid is a typical representation of the perception of
road components, e.g., drivable area, in autonomous driving. Most existing
approaches rely on cameras only to perform segmentation in BEV space, which is
fundamentally constrained by the absence of reliable depth information. The
latest works leverage both camera and LiDAR modalities but suboptimally fuse
their features using simple, concatenation-based mechanisms. In this paper, we
address these problems by enhancing the alignment of the unimodal features in
order to aid feature fusion, as well as enhancing the alignment between the
cameras' perspective view (PV) and BEV representations. We propose X-Align, a
novel end-to-end cross-modal and cross-view learning framework for BEV
segmentation consisting of the following components: (i) a novel Cross-Modal
Feature Alignment (X-FA) loss, (ii) an attention-based Cross-Modal Feature
Fusion (X-FF) module to align multi-modal BEV features implicitly, and (iii) an
auxiliary PV segmentation branch with Cross-View Segmentation Alignment (X-SA)
losses to improve the PV-to-BEV transformation. We evaluate our proposed method
across two commonly used benchmark datasets, i.e., nuScenes and KITTI-360.
Notably, X-Align significantly outperforms the state-of-the-art by 3 absolute
mIoU points on nuScenes. We also provide extensive ablation studies to
demonstrate the effectiveness of the individual components.