Neural Scene Chronology
Haotong Lin Qianqian Wang Ruojin Cai Sida Peng Hadar Averbuch-Elor Xiaowei Zhou Noah Snavely
In this work, we aim to reconstruct a time-varying 3D model, capable of
rendering photo-realistic renderings with independent control of viewpoint,
illumination, and time, from Internet photos of large-scale landmarks. The core
challenges are twofold. First, different types of temporal changes, such as
illumination and changes to the underlying scene itself (such as replacing one
graffiti artwork with another) are entangled together in the imagery. Second,
scene-level temporal changes are often discrete and sporadic over time, rather
than continuous. To tackle these problems, we propose a new scene
representation equipped with a novel temporal step function encoding method
that can model discrete scene-level content changes as piece-wise constant
functions over time. Specifically, we represent the scene as a space-time
radiance field with a per-image illumination embedding, where
temporally-varying scene changes are encoded using a set of learned step
functions. To facilitate our task of chronology reconstruction from Internet
imagery, we also collect a new dataset of four scenes that exhibit various
changes over time. We demonstrate that our method exhibits state-of-the-art
view synthesis results on this dataset, while achieving independent control of
viewpoint, time, and illumination.