GPN: Generative Point-based NeRF
Haipeng Wang
Scanning real-life scenes with modern registration devices typically gives
incomplete point cloud representations, primarily due to the limitations of
partial scanning, 3D occlusions, and dynamic light conditions. Recent works on
processing incomplete point clouds have always focused on point cloud
completion. However, these approaches do not ensure consistency between the
completed point cloud and the captured images regarding color and geometry. We
propose using Generative Point-based NeRF (GPN) to reconstruct and repair a
partial cloud by fully utilizing the scanning images and the corresponding
reconstructed cloud. The repaired point cloud can achieve multi-view
consistency with the captured images at high spatial resolution. For the
finetunes of a single scene, we optimize the global latent condition by
incorporating an Auto-Decoder architecture while retaining multi-view
consistency. As a result, the generated point clouds are smooth, plausible, and
geometrically consistent with the partial scanning images. Extensive
experiments on ShapeNet demonstrate that our works achieve competitive
performances to the other state-of-the-art point cloud-based neural scene
rendering and editing performances.