SVNR: Spatially-variant Noise Removal with Denoising Diffusion
Naama Pearl Yaron Brodsky Dana Berman Assaf Zomet Alex Rav Acha Daniel Cohen-Or Dani Lischinski
Denoising diffusion models have recently shown impressive results in
generative tasks. By learning powerful priors from huge collections of training
images, such models are able to gradually modify complete noise to a clean
natural image via a sequence of small denoising steps, seemingly making them
well-suited for single image denoising. However, effectively applying denoising
diffusion models to removal of realistic noise is more challenging than it may
seem, since their formulation is based on additive white Gaussian noise, unlike
noise in real-world images. In this work, we present SVNR, a novel formulation
of denoising diffusion that assumes a more realistic, spatially-variant noise
model. SVNR enables using the noisy input image as the starting point for the
denoising diffusion process, in addition to conditioning the process on it. To
this end, we adapt the diffusion process to allow each pixel to have its own
time embedding, and propose training and inference schemes that support
spatially-varying time maps. Our formulation also accounts for the correlation
that exists between the condition image and the samples along the modified
diffusion process. In our experiments we demonstrate the advantages of our
approach over a strong diffusion model baseline, as well as over a
state-of-the-art single image denoising method.