GradientSurf: Gradient-Domain Neural Surface Reconstruction from RGB
  Video
Crane He Chen Joerg Liebelt
This paper proposes GradientSurf, a novel algorithm for real time surface
reconstruction from monocular RGB video. Inspired by Poisson Surface
Reconstruction, the proposed method builds on the tight coupling between
surface, volume, and oriented point cloud and solves the reconstruction problem
in gradient-domain. Unlike Poisson Surface Reconstruction which finds an
offline solution to the Poisson equation by solving a linear system after the
scanning process is finished, our method finds online solutions from partial
scans with a neural network incrementally where the Poisson layer is designed
to supervise both local and global reconstruction. The main challenge that
existing methods suffer from when reconstructing from RGB signal is a lack of
details in the reconstructed surface. We hypothesize this is due to the
spectral bias of neural networks towards learning low frequency geometric
features. To address this issue, the reconstruction problem is cast onto
gradient domain, where zeroth-order and first-order energies are minimized. The
zeroth-order term penalizes location of the surface. The first-order term
penalizes the difference between the gradient of reconstructed implicit
function and the vector field formulated from oriented point clouds sampled at
adaptive local densities. For the task of indoor scene reconstruction, visual
and quantitative experimental results show that the proposed method
reconstructs surfaces with more details in curved regions and higher fidelity
for small objects than previous methods.