NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting
  Guidance
Hanlin Chen Chen Li Gim Hee Lee
Existing neural implicit surface reconstruction methods have achieved
impressive performance in multi-view 3D reconstruction by leveraging explicit
geometry priors such as depth maps or point clouds as regularization. However,
the reconstruction results still lack fine details because of the over-smoothed
depth map or sparse point cloud. In this work, we propose a neural implicit
surface reconstruction pipeline with guidance from 3D Gaussian Splatting to
recover highly detailed surfaces. The advantage of 3D Gaussian Splatting is
that it can generate dense point clouds with detailed structure. Nonetheless, a
naive adoption of 3D Gaussian Splatting can fail since the generated points are
the centers of 3D Gaussians that do not necessarily lie on the surface. We thus
introduce a scale regularizer to pull the centers close to the surface by
enforcing the 3D Gaussians to be extremely thin. Moreover, we propose to refine
the point cloud from 3D Gaussians Splatting with the normal priors from the
surface predicted by neural implicit models instead of using a fixed set of
points as guidance. Consequently, the quality of surface reconstruction
improves from the guidance of the more accurate 3D Gaussian splatting. By
jointly optimizing the 3D Gaussian Splatting and the neural implicit model, our
approach benefits from both representations and generates complete surfaces
with intricate details. Experiments on Tanks and Temples verify the
effectiveness of our proposed method.