Neural Relighting with Subsurface Scattering by Learning the Radiance
  Transfer Gradient
Shizhan Zhu Shunsuke Saito Aljaz Bozic Carlos Aliaga Trevor Darrell Christoph Lassner
Reconstructing and relighting objects and scenes under varying lighting
conditions is challenging: existing neural rendering methods often cannot
handle the complex interactions between materials and light. Incorporating
pre-computed radiance transfer techniques enables global illumination, but
still struggles with materials with subsurface scattering effects. We propose a
novel framework for learning the radiance transfer field via volume rendering
and utilizing various appearance cues to refine geometry end-to-end. This
framework extends relighting and reconstruction capabilities to handle a wider
range of materials in a data-driven fashion. The resulting models produce
plausible rendering results in existing and novel conditions. We will release
our code and a novel light stage dataset of objects with subsurface scattering
effects publicly available.