Denoising Monte Carlo Renders With Diffusion Models
Vaibhav Vavilala Rahul Vasanth David Forsyth
Physically-based renderings contain Monte-Carlo noise, with variance that
increases as the number of rays per pixel decreases. This noise, while
zero-mean for good modern renderers, can have heavy tails (most notably, for
scenes containing specular or refractive objects). Learned methods for
restoring low fidelity renders are highly developed, because suppressing render
noise means one can save compute and use fast renders with few rays per pixel.
We demonstrate that a diffusion model can denoise low fidelity renders
successfully. Furthermore, our method can be conditioned on a variety of
natural render information, and this conditioning helps performance.
Quantitative experiments show that our method is competitive with SOTA across a
range of sampling rates, but current metrics slightly favor competitor methods.
Qualitative examination of the reconstructions suggests that the metrics
themselves may not be reliable. The image prior applied by a diffusion method
strongly favors reconstructions that are "like" real images -- so have straight
shadow boundaries, curved specularities, no "fireflies" and the like -- and
metrics do not account for this. We show numerous examples where methods
preferred by current metrics produce qualitatively weaker reconstructions than
ours.