OccFusion: Depth Estimation Free Multi-sensor Fusion for 3D Occupancy
  Prediction
Ji Zhang Yiran Ding
3D occupancy prediction based on multi-sensor fusion, crucial for a reliable
autonomous driving system, enables fine-grained understanding of 3D scenes.
Previous fusion-based 3D occupancy predictions relied on depth estimation for
processing 2D image features. However, depth estimation is an ill-posed
problem, hindering the accuracy and robustness of these methods. Furthermore,
fine-grained occupancy prediction demands extensive computational resources. We
introduce OccFusion, a multi-modal fusion method free from depth estimation,
and a corresponding point cloud sampling algorithm for dense integration of
image features. Building on this, we propose an active training method and an
active coarse to fine pipeline, enabling the model to adaptively learn more
from complex samples and optimize predictions specifically for challenging
areas such as small or overlapping objects. The active methods we propose can
be naturally extended to any occupancy prediction model. Experiments on the
OpenOccupancy benchmark show our method surpasses existing state-of-the-art
(SOTA) multi-modal methods in IoU across all categories. Additionally, our
model is more efficient during both the training and inference phases,
requiring far fewer computational resources. Comprehensive ablation studies
demonstrate the effectiveness of our proposed techniques.