InverseMatrixVT3D: An Efficient Projection Matrix-Based Approach for 3D
  Occupancy Prediction
Zhenxing Ming Julie Stephany Berrio Mao Shan Stewart Worrall
This paper introduces InverseMatrixVT3D, an efficient method for transforming
multi-view image features into 3D feature volumes for 3D semantic occupancy
prediction. Existing methods for constructing 3D volumes often rely on depth
estimation, device-specific operators, or transformer queries, which hinders
the widespread adoption of 3D occupancy models. In contrast, our approach
leverages two projection matrices to store the static mapping relationships and
matrix multiplications to efficiently generate global Bird's Eye View (BEV)
features and local 3D feature volumes. Specifically, we achieve this by
performing matrix multiplications between multi-view image feature maps and two
sparse projection matrices. We introduce a sparse matrix handling technique for
the projection matrices to optimise GPU memory usage. Moreover, a global-local
attention fusion module is proposed to integrate the global BEV features with
the local 3D feature volumes to obtain the final 3D volume. We also employ a
multi-scale supervision mechanism to further enhance performance. Comprehensive
experiments on the nuScenes dataset demonstrate the simplicity and
effectiveness of our method. The code will be made available
at:https://github.com/DanielMing123/InverseMatrixVT3D