ClusterDDPM: An EM clustering framework with Denoising Diffusion
  Probabilistic Models
Jie Yan Jing Liu Zhong-yuan Zhang
Variational autoencoder (VAE) and generative adversarial networks (GAN) have
found widespread applications in clustering and have achieved significant
success. However, the potential of these approaches may be limited due to VAE's
mediocre generation capability or GAN's well-known instability during
adversarial training. In contrast, denoising diffusion probabilistic models
(DDPMs) represent a new and promising class of generative models that may
unlock fresh dimensions in clustering. In this study, we introduce an
innovative expectation-maximization (EM) framework for clustering using DDPMs.
In the E-step, we aim to derive a mixture of Gaussian priors for the subsequent
M-step. In the M-step, our focus lies in learning clustering-friendly latent
representations for the data by employing the conditional DDPM and matching the
distribution of latent representations to the mixture of Gaussian priors. We
present a rigorous theoretical analysis of the optimization process in the
M-step, proving that the optimizations are equivalent to maximizing the lower
bound of the Q function within the vanilla EM framework under certain
constraints. Comprehensive experiments validate the advantages of the proposed
framework, showcasing superior performance in clustering, unsupervised
conditional generation and latent representation learning.