SiLVR: Scalable Lidar-Visual Reconstruction with Neural Radiance Fields
  for Robotic Inspection
Yifu Tao Yash Bhalgat Lanke Frank Tarimo Fu Matias Mattamala Nived Chebrolu Maurice Fallon
We present a neural-field-based large-scale reconstruction system that fuses
lidar and vision data to generate high-quality reconstructions that are
geometrically accurate and capture photo-realistic textures. This system adapts
the state-of-the-art neural radiance field (NeRF) representation to also
incorporate lidar data which adds strong geometric constraints on the depth and
surface normals. We exploit the trajectory from a real-time lidar SLAM system
to bootstrap a Structure-from-Motion (SfM) procedure to both significantly
reduce the computation time and to provide metric scale which is crucial for
lidar depth loss. We use submapping to scale the system to large-scale
environments captured over long trajectories. We demonstrate the reconstruction
system with data from a multi-camera, lidar sensor suite onboard a legged
robot, hand-held while scanning building scenes for 600 metres, and onboard an
aerial robot surveying a multi-storey mock disaster site-building. Website:
https://ori-drs.github.io/projects/silvr/