CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect
  Camera Poses
Hongyu Fu Xin Yu Lincheng Li Li Zhang
Existing volumetric neural rendering techniques, such as Neural Radiance
Fields (NeRF), face limitations in synthesizing high-quality novel views when
the camera poses of input images are imperfect. To address this issue, we
propose a novel 3D reconstruction framework that enables simultaneous
optimization of camera poses, dubbed CBARF (Cascaded Bundle-Adjusting NeRF).In
a nutshell, our framework optimizes camera poses in a coarse-to-fine manner and
then reconstructs scenes based on the rectified poses. It is observed that the
initialization of camera poses has a significant impact on the performance of
bundle-adjustment (BA). Therefore, we cascade multiple BA modules at different
scales to progressively improve the camera poses. Meanwhile, we develop a
neighbor-replacement strategy to further optimize the results of BA in each
stage. In this step, we introduce a novel criterion to effectively identify
poorly estimated camera poses. Then we replace them with the poses of
neighboring cameras, thus further eliminating the impact of inaccurate camera
poses. Once camera poses have been optimized, we employ a density voxel grid to
generate high-quality 3D reconstructed scenes and images in novel views.
Experimental results demonstrate that our CBARF model achieves state-of-the-art
performance in both pose optimization and novel view synthesis, especially in
the existence of large camera pose noise.