SO-NeRF: Active View Planning for NeRF using Surrogate Objectives
Keifer Lee Shubham Gupta Sunglyoung Kim Bhargav Makwana Chao Chen Chen Feng
Despite the great success of Neural Radiance Fields (NeRF), its
data-gathering process remains vague with only a general rule of thumb of
sampling as densely as possible. The lack of understanding of what actually
constitutes good views for NeRF makes it difficult to actively plan a sequence
of views that yield the maximal reconstruction quality. We propose Surrogate
Objectives for Active Radiance Fields (SOAR), which is a set of interpretable
functions that evaluates the goodness of views using geometric and photometric
visual cues - surface coverage, geometric complexity, textural complexity, and
ray diversity. Moreover, by learning to infer the SOAR scores from a deep
network, SOARNet, we are able to effectively select views in mere seconds
instead of hours, without the need for prior visits to all the candidate views
or training any radiance field during such planning. Our experiments show
SOARNet outperforms the baselines with $\sim$80x speed-up while achieving
better or comparable reconstruction qualities. We finally show that SOAR is
model-agnostic, thus it generalizes across fully neural-implicit to fully
explicit approaches.