RANRAC: Robust Neural Scene Representations via Random Ray Consensus
Benno Buschmann Andreea Dogaru Elmar Eisemann Michael Weinmann Bernhard Egger
We introduce RANRAC, a robust reconstruction algorithm for 3D objects
handling occluded and distracted images, which is a particularly challenging
scenario that prior robust reconstruction methods cannot deal with. Our
solution supports single-shot reconstruction by involving light-field networks,
and is also applicable to photo-realistic, robust, multi-view reconstruction
from real-world images based on neural radiance fields. While the algorithm
imposes certain limitations on the scene representation and, thereby, the
supported scene types, it reliably detects and excludes inconsistent
perspectives, resulting in clean images without floating artifacts. Our
solution is based on a fuzzy adaption of the random sample consensus paradigm,
enabling its application to large scale models. We interpret the minimal number
of samples to determine the model parameters as a tunable hyperparameter. This
is applicable, as a cleaner set of samples improves reconstruction quality.
Further, this procedure also handles outliers. Especially for conditioned
models, it can result in the same local minimum in the latent space as would be
obtained with a completely clean set. We report significant improvements for
novel-view synthesis in occluded scenarios, of up to 8dB PSNR compared to the
baseline.