Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian
  Mixture Models
Yuchen Wu Minshuo Chen Zihao Li Mengdi Wang Yuting Wei
Diffusion models benefit from instillation of task-specific information into
the score function to steer the sample generation towards desired properties.
Such information is coined as guidance. For example, in text-to-image
synthesis, text input is encoded as guidance to generate semantically aligned
images. Proper guidance inputs are closely tied to the performance of diffusion
models. A common observation is that strong guidance promotes a tight alignment
to the task-specific information, while reducing the diversity of the generated
samples. In this paper, we provide the first theoretical study towards
understanding the influence of guidance on diffusion models in the context of
Gaussian mixture models. Under mild conditions, we prove that incorporating
diffusion guidance not only boosts classification confidence but also
diminishes distribution diversity, leading to a reduction in the differential
entropy of the output distribution. Our analysis covers the widely adopted
sampling schemes including DDPM and DDIM, and leverages comparison inequalities
for differential equations as well as the Fokker-Planck equation that
characterizes the evolution of probability density function, which may be of
independent theoretical interest.