Natural language guidance of high-fidelity text-to-speech with synthetic
  annotations
Dan Lyth Simon King
Text-to-speech models trained on large-scale datasets have demonstrated
impressive in-context learning capabilities and naturalness. However, control
of speaker identity and style in these models typically requires conditioning
on reference speech recordings, limiting creative applications. Alternatively,
natural language prompting of speaker identity and style has demonstrated
promising results and provides an intuitive method of control. However,
reliance on human-labeled descriptions prevents scaling to large datasets. Our
work bridges the gap between these two approaches. We propose a scalable method
for labeling various aspects of speaker identity, style, and recording
conditions. We then apply this method to a 45k hour dataset, which we use to
train a speech language model. Furthermore, we propose simple methods for
increasing audio fidelity, significantly outperforming recent work despite
relying entirely on found data. Our results demonstrate high-fidelity speech
generation in a diverse range of accents, prosodic styles, channel conditions,
and acoustic conditions, all accomplished with a single model and intuitive
natural language conditioning. Audio samples can be heard at
https://text-description-to-speech.com/.