SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object
  Detection
Hongcheng Zhang Liu Liang Pengxin Zeng Xiao Song Zhe Wang
Sparse 3D detectors have received significant attention since the query-based
paradigm embraces low latency without explicit dense BEV feature construction.
However, these detectors achieve worse performance than their dense
counterparts. In this paper, we find the key to bridging the performance gap is
to enhance the awareness of rich representations in two modalities. Here, we
present a high-performance fully sparse detector for end-to-end multi-modality
3D object detection. The detector, termed SparseLIF, contains three key
designs, which are (1) Perspective-Aware Query Generation (PAQG) to generate
high-quality 3D queries with perspective priors, (2) RoI-Aware Sampling (RIAS)
to further refine prior queries by sampling RoI features from each modality,
(3) Uncertainty-Aware Fusion (UAF) to precisely quantify the uncertainty of
each sensor modality and adaptively conduct final multi-modality fusion, thus
achieving great robustness against sensor noises. By the time of submission
(2024/03/08), SparseLIF achieves state-of-the-art performance on the nuScenes
dataset, ranking 1st on both validation set and test benchmark, outperforming
all state-of-the-art 3D object detectors by a notable margin. The source code
will be released upon acceptance.