Tiled Multiplane Images for Practical 3D Photography
Numair Khan Douglas Lanman Lei Xiao
The task of synthesizing novel views from a single image has useful
applications in virtual reality and mobile computing, and a number of
approaches to the problem have been proposed in recent years. A Multiplane
Image (MPI) estimates the scene as a stack of RGBA layers, and can model
complex appearance effects, anti-alias depth errors and synthesize soft edges
better than methods that use textured meshes or layered depth images. And
unlike neural radiance fields, an MPI can be efficiently rendered on graphics
hardware. However, MPIs are highly redundant and require a large number of
depth layers to achieve plausible results. Based on the observation that the
depth complexity in local image regions is lower than that over the entire
image, we split an MPI into many small, tiled regions, each with only a few
depth planes. We call this representation a Tiled Multiplane Image (TMPI). We
propose a method for generating a TMPI with adaptive depth planes for
single-view 3D photography in the wild. Our synthesized results are comparable
to state-of-the-art single-view MPI methods while having lower computational
overhead.