Unsupervised speech enhancement with diffusion-based generative models
Berné Nortier Mostafa Sadeghi Romain Serizel
Recently, conditional score-based diffusion models have gained significant
attention in the field of supervised speech enhancement, yielding
state-of-the-art performance. However, these methods may face challenges when
generalising to unseen conditions. To address this issue, we introduce an
alternative approach that operates in an unsupervised manner, leveraging the
generative power of diffusion models. Specifically, in a training phase, a
clean speech prior distribution is learnt in the short-time Fourier transform
(STFT) domain using score-based diffusion models, allowing it to
unconditionally generate clean speech from Gaussian noise. Then, we develop a
posterior sampling methodology for speech enhancement by combining the learnt
clean speech prior with a noise model for speech signal inference. The noise
parameters are simultaneously learnt along with clean speech estimation through
an iterative expectationmaximisation (EM) approach. To the best of our
knowledge, this is the first work exploring diffusion-based generative models
for unsupervised speech enhancement, demonstrating promising results compared
to a recent variational auto-encoder (VAE)-based unsupervised approach and a
state-of-the-art diffusion-based supervised method. It thus opens a new
direction for future research in unsupervised speech enhancement.