NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation
Rui Yu Jiachen Liu Zihan Zhou Sharon X. Huang
In various applications, such as robotic navigation and remote visual
assistance, expanding the field of view (FOV) of the camera proves beneficial
for enhancing environmental perception. Unlike image outpainting techniques
aimed solely at generating aesthetically pleasing visuals, these applications
demand an extended view that faithfully represents the scene. To achieve this,
we formulate a new problem of faithful FOV extrapolation that utilizes a set of
pre-captured images as prior knowledge of the scene. To address this problem,
we present a simple yet effective solution called NeRF-Enhanced Outpainting
(NEO) that uses extended-FOV images generated through NeRF to train a
scene-specific image outpainting model. To assess the performance of NEO, we
conduct comprehensive evaluations on three photorealistic datasets and one
real-world dataset. Extensive experiments on the benchmark datasets showcase
the robustness and potential of our method in addressing this challenge. We
believe our work lays a strong foundation for future exploration within the
research community.