MCMC-Correction of Score-Based Diffusion Models for Model Composition
Anders Sjöberg Jakob Lindqvist Magnus Önnheim Mats Jirstrand Lennart Svensson
Diffusion models can be parameterised in terms of either a score or an energy
function. The energy parameterisation has better theoretical properties, mainly
that it enables an extended sampling procedure with a Metropolis--Hastings
correction step, based on the change in total energy in the proposed samples.
However, it seems to yield slightly worse performance, and more importantly,
due to the widespread popularity of score-based diffusion, there are limited
availability of off-the-shelf pre-trained energy-based ones. This limitation
undermines the purpose of model composition, which aims to combine pre-trained
models to sample from new distributions. Our proposal, however, suggests
retaining the score parameterization and instead computing the energy-based
acceptance probability through line integration of the score function. This
allows us to re-use existing diffusion models and still combine the reverse
process with various Markov-Chain Monte Carlo (MCMC) methods. We evaluate our
method on a 2D experiment and find that it achieve similar or arguably better
performance than the energy parameterisation.