Volumetric Cloud Field Reconstruction
Jacob Lin Miguel Farinha Edward Gryspeerdt Ronald Clark
Volumetric phenomena, such as clouds and fog, present a significant challenge
for 3D reconstruction systems due to their translucent nature and their complex
interactions with light. Conventional techniques for reconstructing scattering
volumes rely on controlled setups, limiting practical applications. This paper
introduces an approach to reconstructing volumes from a few input stereo pairs.
We propose a novel deep learning framework that integrates a deep stereo model
with a 3D Convolutional Neural Network (3D CNN) and an advection module,
capable of capturing the shape and dynamics of volumes. The stereo depths are
used to carve empty space around volumes, providing the 3D CNN with a prior for
coping with the lack of input views. Refining our output, the advection module
leverages the temporal evolution of the medium, providing a mechanism to infer
motion and improve temporal consistency. The efficacy of our system is
demonstrated through its ability to estimate density and velocity fields of
large-scale volumes, in this case, clouds, from a sparse set of stereo image
pairs.