Diffusion Models With Learned Adaptive Noise
Subham Sekhar Sahoo Aaron Gokaslan Chris De Sa Volodymyr Kuleshov
Diffusion models have gained traction as powerful algorithms for synthesizing
high-quality images. Central to these algorithms is the diffusion process,
which maps data to noise according to equations inspired by thermodynamics and
can significantly impact performance. A widely held assumption is that the ELBO
objective of a diffusion model is invariant to the noise process (Kingma et
al.,2021). In this work, we dispel this assumption -- we propose multivariate
learned adaptive noise (MuLAN), a learned diffusion process that applies
Gaussian noise at different rates across an image. Our method consists of three
components -- a multivariate noise schedule, instance-conditional diffusion,
and auxiliary variables -- which ensure that the learning objective is no
longer invariant to the choice of the noise schedule as in previous works. Our
work is grounded in Bayesian inference and casts the learned diffusion process
as an approximate variational posterior that yields a tighter lower bound on
marginal likelihood. Empirically, MuLAN sets a new state-of-the-art in density
estimation on CIFAR-10 and ImageNet compared to classical diffusion. Code is
available at https://github.com/s-sahoo/MuLAN