Investigating the Effect of Language Models in Sequence Discriminative
  Training for Neural Transducers
Zijian Yang Wei Zhou Ralf Schlüter Hermann Ney
In this work, we investigate the effect of language models (LMs) with
different context lengths and label units (phoneme vs. word) used in sequence
discriminative training for phoneme-based neural transducers. Both lattice-free
and N-best-list approaches are examined. For lattice-free methods with
phoneme-level LMs, we propose a method to approximate the context history to
employ LMs with full-context dependency. This approximation can be extended to
arbitrary context length and enables the usage of word-level LMs in
lattice-free methods. Moreover, a systematic comparison is conducted across
lattice-free and N-best-list-based methods. Experimental results on Librispeech
show that using the word-level LM in training outperforms the phoneme-level LM.
Besides, we find that the context size of the LM used for probability
computation has a limited effect on performance. Moreover, our results reveal
the pivotal importance of the hypothesis space quality in sequence
discriminative training.