Towards Monocular Shape from Refraction
Antonin Sulc Imari Sato Bastian Goldluecke Tali Treibitz
Refraction is a common physical phenomenon and has long been researched in
computer vision. Objects imaged through a refractive object appear distorted in
the image as a function of the shape of the interface between the media. This
hinders many computer vision applications, but can be utilized for obtaining
the geometry of the refractive interface. Previous approaches for refractive
surface recovery largely relied on various priors or additional information
like multiple images of the analyzed surface. In contrast, we claim that a
simple energy function based on Snell's law enables the reconstruction of an
arbitrary refractive surface geometry using just a single image and known
background texture and geometry. In the case of a single point, Snell's law has
two degrees of freedom, therefore to estimate a surface depth, we need
additional information. We show that solving for an entire surface at once
introduces implicit parameter-free spatial regularization and yields convincing
results when an intelligent initial guess is provided. We demonstrate our
approach through simulations and real-world experiments, where the
reconstruction shows encouraging results in the single-frame monocular setting.