Flexible Techniques for Differentiable Rendering with 3D Gaussians
Leonid Keselman Martial Hebert
Fast, reliable shape reconstruction is an essential ingredient in many
computer vision applications. Neural Radiance Fields demonstrated that
photorealistic novel view synthesis is within reach, but was gated by
performance requirements for fast reconstruction of real scenes and objects.
Several recent approaches have built on alternative shape representations, in
particular, 3D Gaussians. We develop extensions to these renderers, such as
integrating differentiable optical flow, exporting watertight meshes and
rendering per-ray normals. Additionally, we show how two of the recent methods
are interoperable with each other. These reconstructions are quick, robust, and
easily performed on GPU or CPU. For code and visual examples, see
https://leonidk.github.io/fmb-plus