RL-based Stateful Neural Adaptive Sampling and Denoising for Real-Time
  Path Tracing
Antoine Scardigli Lukas Cavigelli Lorenz K. Müller
Monte-Carlo path tracing is a powerful technique for realistic image
synthesis but suffers from high levels of noise at low sample counts, limiting
its use in real-time applications. To address this, we propose a framework with
end-to-end training of a sampling importance network, a latent space encoder
network, and a denoiser network. Our approach uses reinforcement learning to
optimize the sampling importance network, thus avoiding explicit numerically
approximated gradients. Our method does not aggregate the sampled values per
pixel by averaging but keeps all sampled values which are then fed into the
latent space encoder. The encoder replaces handcrafted spatiotemporal
heuristics by learned representations in a latent space. Finally, a neural
denoiser is trained to refine the output image. Our approach increases visual
quality on several challenging datasets and reduces rendering times for equal
quality by a factor of 1.6x compared to the previous state-of-the-art, making
it a promising solution for real-time applications.