On the Relevance of Phoneme Duration Variability of Synthesized Training
  Data for Automatic Speech Recognition
Nick Rossenbach Benedikt Hilmes Ralf Schlüter
Synthetic data generated by text-to-speech (TTS) systems can be used to
improve automatic speech recognition (ASR) systems in low-resource or domain
mismatch tasks. It has been shown that TTS-generated outputs still do not have
the same qualities as real data. In this work we focus on the temporal
structure of synthetic data and its relation to ASR training. By using a novel
oracle setup we show how much the degradation of synthetic data quality is
influenced by duration modeling in non-autoregressive (NAR) TTS. To get
reference phoneme durations we use two common alignment methods, a hidden
Markov Gaussian-mixture model (HMM-GMM) aligner and a neural connectionist
temporal classification (CTC) aligner. Using a simple algorithm based on random
walks we shift phoneme duration distributions of the TTS system closer to real
durations, resulting in an improvement of an ASR system using synthetic data in
a semi-supervised setting.