FB-OCC: 3D Occupancy Prediction based on Forward-Backward View
  Transformation
Zhiqi Li Zhiding Yu David Austin Mingsheng Fang Shiyi Lan Jan Kautz Jose M. Alvarez
This technical report summarizes the winning solution for the 3D Occupancy
Prediction Challenge, which is held in conjunction with the CVPR 2023 Workshop
on End-to-End Autonomous Driving and CVPR 23 Workshop on Vision-Centric
Autonomous Driving Workshop. Our proposed solution FB-OCC builds upon FB-BEV, a
cutting-edge camera-based bird's-eye view perception design using
forward-backward projection. On top of FB-BEV, we further study novel designs
and optimization tailored to the 3D occupancy prediction task, including joint
depth-semantic pre-training, joint voxel-BEV representation, model scaling up,
and effective post-processing strategies. These designs and optimization result
in a state-of-the-art mIoU score of 54.19% on the nuScenes dataset, ranking the
1st place in the challenge track. Code and models will be released at:
https://github.com/NVlabs/FB-BEV.