Volumetric Rendering with Baked Quadrature Fields
Gopal Sharma Daniel Rebain Kwang Moo Yi Andrea Tagliasacchi
We propose a novel Neural Radiance Field (NeRF) representation for non-opaque
scenes that allows fast inference by utilizing textured polygons. Despite the
high-quality novel view rendering that NeRF provides, a critical limitation is
that it relies on volume rendering that can be computationally expensive and
does not utilize the advancements in modern graphics hardware. Existing methods
for this problem fall short when it comes to modelling volumetric effects as
they rely purely on surface rendering. We thus propose to model the scene with
polygons, which can then be used to obtain the quadrature points required to
model volumetric effects, and also their opacity and colour from the texture.
To obtain such polygonal mesh, we train a specialized field whose
zero-crossings would correspond to the quadrature points when volume rendering,
and perform marching cubes on this field. We then rasterize the polygons and
utilize the fragment shaders to obtain the final colour image. Our method
allows rendering on various devices and easy integration with existing graphics
frameworks while keeping the benefits of volume rendering alive.