Relightable Neural Assets
Krishna Mullia Fujun Luan Xin Sun Miloš Hašan
High-fidelity 3D assets with materials composed of fibers (including hair),
complex layered material shaders, or fine scattering geometry are ubiquitous in
high-end realistic rendering applications. Rendering such models is
computationally expensive due to heavy shaders and long scattering paths.
Moreover, implementing the shading and scattering models is non-trivial and has
to be done not only in the 3D content authoring software (which is necessarily
complex), but also in all downstream rendering solutions. For example, web and
mobile viewers for complex 3D assets are desirable, but frequently cannot
support the full shading complexity allowed by the authoring application. Our
goal is to design a neural representation for 3D assets with complex shading
that supports full relightability and full integration into existing renderers.
We provide an end-to-end shading solution at the first intersection of a ray
with the underlying geometry. All shading and scattering is precomputed and
included in the neural asset; no multiple scattering paths need to be traced,
and no complex shading models need to be implemented to render our assets,
beyond a single neural architecture. We combine an MLP decoder with a feature
grid. Shading consists of querying a feature vector, followed by an MLP
evaluation producing the final reflectance value. Our method provides
high-fidelity shading, close to the ground-truth Monte Carlo estimate even at
close-up views. We believe our neural assets could be used in practical
renderers, providing significant speed-ups and simplifying renderer
implementations.