360-Degree Panorama Generation from Few Unregistered NFoV Images
Jionghao Wang Ziyu Chen Jun Ling Rong Xie Li Song
360$^\circ$ panoramas are extensively utilized as environmental light sources
in computer graphics. However, capturing a 360$^\circ$ $\times$ 180$^\circ$
panorama poses challenges due to the necessity of specialized and costly
equipment, and additional human resources. Prior studies develop various
learning-based generative methods to synthesize panoramas from a single Narrow
Field-of-View (NFoV) image, but they are limited in alterable input patterns,
generation quality, and controllability. To address these issues, we propose a
novel pipeline called PanoDiff, which efficiently generates complete
360$^\circ$ panoramas using one or more unregistered NFoV images captured from
arbitrary angles. Our approach has two primary components to overcome the
limitations. Firstly, a two-stage angle prediction module to handle various
numbers of NFoV inputs. Secondly, a novel latent diffusion-based panorama
generation model uses incomplete panorama and text prompts as control signals
and utilizes several geometric augmentation schemes to ensure geometric
properties in generated panoramas. Experiments show that PanoDiff achieves
state-of-the-art panoramic generation quality and high controllability, making
it suitable for applications such as content editing.