NeRF2Points: Large-Scale Point Cloud Generation From Street Views'
  Radiance Field Optimization
Peng Tu Xun Zhou Mingming Wang Xiaojun Yang Bo Peng Ping Chen Xiu Su Yawen Huang Yefeng Zheng Chang Xu
Neural Radiance Fields (NeRF) have emerged as a paradigm-shifting methodology
for the photorealistic rendering of objects and environments, enabling the
synthesis of novel viewpoints with remarkable fidelity. This is accomplished
through the strategic utilization of object-centric camera poses characterized
by significant inter-frame overlap. This paper explores a compelling,
alternative utility of NeRF: the derivation of point clouds from aggregated
urban landscape imagery. The transmutation of street-view data into point
clouds is fraught with complexities, attributable to a nexus of interdependent
variables. First, high-quality point cloud generation hinges on precise camera
poses, yet many datasets suffer from inaccuracies in pose metadata. Also, the
standard approach of NeRF is ill-suited for the distinct characteristics of
street-view data from autonomous vehicles in vast, open settings. Autonomous
vehicle cameras often record with limited overlap, leading to blurring,
artifacts, and compromised pavement representation in NeRF-based point clouds.
In this paper, we present NeRF2Points, a tailored NeRF variant for urban point
cloud synthesis, notable for its high-quality output from RGB inputs alone. Our
paper is supported by a bespoke, high-resolution 20-kilometer urban street
dataset, designed for point cloud generation and evaluation. NeRF2Points
adeptly navigates the inherent challenges of NeRF-based point cloud synthesis
through the implementation of the following strategic innovations: (1)
Integration of Weighted Iterative Geometric Optimization (WIGO) and Structure
from Motion (SfM) for enhanced camera pose accuracy, elevating street-view data
precision. (2) Layered Perception and Integrated Modeling (LPiM) is designed
for distinct radiance field modeling in urban environments, resulting in
coherent point cloud representations.