Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion
Ziyuan Qu Omkar Vengurlekar Mohamad Qadri Kevin Zhang Michael Kaess Christopher Metzler Suren Jayasuriya Adithya Pediredla
Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent
technique in computer vision and graphics for reconstructing 3D scenes. GS
represents a scene as a set of 3D Gaussians with varying opacities and employs
a computationally efficient splatting operation along with analytical
derivatives to compute the 3D Gaussian parameters given scene images captured
from various viewpoints. Unfortunately, capturing surround view ($360^{\circ}$
viewpoint) images is impossible or impractical in many real-world imaging
scenarios, including underwater imaging, rooms inside a building, and
autonomous navigation. In these restricted baseline imaging scenarios, the GS
algorithm suffers from a well-known 'missing cone' problem, which results in
poor reconstruction along the depth axis. In this manuscript, we demonstrate
that using transient data (from sonars) allows us to address the missing cone
problem by sampling high-frequency data along the depth axis. We extend the
Gaussian splatting algorithms for two commonly used sonars and propose fusion
algorithms that simultaneously utilize RGB camera data and sonar data. Through
simulations, emulations, and hardware experiments across various imaging
scenarios, we show that the proposed fusion algorithms lead to significantly
better novel view synthesis (5 dB improvement in PSNR) and 3D geometry
reconstruction (60% lower Chamfer distance).