SC-NeRF: Self-Correcting Neural Radiance Field with Sparse Views
Liang Song Guangming Wang Jiuming Liu Zhenyang Fu Yanzi Miao Hesheng
In recent studies, the generalization of neural radiance fields for novel
view synthesis task has been widely explored. However, existing methods are
limited to objects and indoor scenes. In this work, we extend the
generalization task to outdoor scenes, trained only on object-level datasets.
This approach presents two challenges. Firstly, the significant distributional
shift between training and testing scenes leads to black artifacts in rendering
results. Secondly, viewpoint changes in outdoor scenes cause ghosting or
missing regions in rendered images. To address these challenges, we propose a
geometric correction module and an appearance correction module based on
multi-head attention mechanisms. We normalize rendered depth and combine it
with light direction as query in the attention mechanism. Our network
effectively corrects varying scene structures and geometric features in outdoor
scenes, generalizing well from object-level to unseen outdoor scenes.
Additionally, we use appearance correction module to correct appearance
features, preventing rendering artifacts like blank borders and ghosting due to
viewpoint changes. By combining these modules, our approach successfully
tackles the challenges of outdoor scene generalization, producing high-quality
rendering results. When evaluated on four datasets (Blender, DTU, LLFF,
Spaces), our network outperforms previous methods. Notably, compared to
MVSNeRF, our network improves average PSNR from 19.369 to 25.989, SSIM from
0.838 to 0.889, and reduces LPIPS from 0.265 to 0.224 on Spaces outdoor scenes.