NeRFuser: Large-Scale Scene Representation by NeRF Fusion
Jiading Fang Shengjie Lin Igor Vasiljevic Vitor Guizilini Rares Ambrus Adrien Gaidon Gregory Shakhnarovich Matthew R. Walter
A practical benefit of implicit visual representations like Neural Radiance
Fields (NeRFs) is their memory efficiency: large scenes can be efficiently
stored and shared as small neural nets instead of collections of images.
However, operating on these implicit visual data structures requires extending
classical image-based vision techniques (e.g., registration, blending) from
image sets to neural fields. Towards this goal, we propose NeRFuser, a novel
architecture for NeRF registration and blending that assumes only access to
pre-generated NeRFs, and not the potentially large sets of images used to
generate them. We propose registration from re-rendering, a technique to infer
the transformation between NeRFs based on images synthesized from individual
NeRFs. For blending, we propose sample-based inverse distance weighting to
blend visual information at the ray-sample level. We evaluate NeRFuser on
public benchmarks and a self-collected object-centric indoor dataset, showing
the robustness of our method, including to views that are challenging to render
from the individual source NeRFs.