ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for
  Sparse View Synthesis
Shoukang Hu Kaichen Zhou Kaiyu Li Longhui Yu Lanqing Hong Tianyang Hu Zhenguo Li Gim Hee Lee Ziwei Liu
Neural Radiance Fields (NeRF) has demonstrated remarkable 3D reconstruction
capabilities with dense view images. However, its performance significantly
deteriorates under sparse view settings. We observe that learning the 3D
consistency of pixels among different views is crucial for improving
reconstruction quality in such cases. In this paper, we propose ConsistentNeRF,
a method that leverages depth information to regularize both multi-view and
single-view 3D consistency among pixels. Specifically, ConsistentNeRF employs
depth-derived geometry information and a depth-invariant loss to concentrate on
pixels that exhibit 3D correspondence and maintain consistent depth
relationships. Extensive experiments on recent representative works reveal that
our approach can considerably enhance model performance in sparse view
conditions, achieving improvements of up to 94% in PSNR, 76% in SSIM, and 31%
in LPIPS compared to the vanilla baselines across various benchmarks, including
DTU, NeRF Synthetic, and LLFF.