Diffusion Models for Audio Restoration
Jean-Marie Lemercier Julius Richter Simon Welker Eloi Moliner Vesa Välimäki Timo Gerkmann
With the development of audio playback devices and fast data transmission,
the demand for high sound quality is rising, for both entertainment and
communications. In this quest for better sound quality, challenges emerge from
distortions and interferences originating at the recording side or caused by an
imperfect transmission pipeline. To address this problem, audio restoration
methods aim to recover clean sound signals from the corrupted input data. We
present here audio restoration algorithms based on diffusion models, with a
focus on speech enhancement and music restoration tasks. Traditional
approaches, often grounded in handcrafted rules and statistical heuristics,
have shaped our understanding of audio signals. In the past decades, there has
been a notable shift towards data-driven methods that exploit the modeling
capabilities of deep neural networks (DNNs). Deep generative models, and among
them diffusion models, have emerged as powerful techniques for learning complex
data distributions. However, relying solely on DNN-based learning approaches
carries the risk of reducing interpretability, particularly when employing
end-to-end models. Nonetheless, data-driven approaches allow more flexibility
in comparison to statistical model-based frameworks whose performance depends
on distributional and statistical assumptions that can be difficult to
guarantee. Here, we aim to show that diffusion models can combine the best of
both worlds and offer the opportunity to design audio restoration algorithms
with a good degree of interpretability and a remarkable performance in terms of
sound quality.