Energy-Based Models For Speech Synthesis
Wanli Sun Zehai Tu Anton Ragni
Recently there has been a lot of interest in non-autoregressive (non-AR)
models for speech synthesis, such as FastSpeech 2 and diffusion models. Unlike
AR models, these models do not have autoregressive dependencies among outputs
which makes inference efficient. This paper expands the range of available
non-AR models with another member called energy-based models (EBMs). The paper
describes how noise contrastive estimation, which relies on the comparison
between positive and negative samples, can be used to train EBMs. It proposes a
number of strategies for generating effective negative samples, including using
high-performing AR models. It also describes how sampling from EBMs can be
performed using Langevin Markov Chain Monte-Carlo (MCMC). The use of Langevin
MCMC enables to draw connections between EBMs and currently popular diffusion
models. Experiments on LJSpeech dataset show that the proposed approach offers
improvements over Tacotron 2.