Learning from Flawed Data: Weakly Supervised Automatic Speech
  Recognition
Dongji Gao Hainan Xu Desh Raj Leibny Paola Garcia Perera Daniel Povey Sanjeev Khudanpur
Training automatic speech recognition (ASR) systems requires large amounts of
well-curated paired data. However, human annotators usually perform
"non-verbatim" transcription, which can result in poorly trained models. In
this paper, we propose Omni-temporal Classification (OTC), a novel training
criterion that explicitly incorporates label uncertainties originating from
such weak supervision. This allows the model to effectively learn speech-text
alignments while accommodating errors present in the training transcripts. OTC
extends the conventional CTC objective for imperfect transcripts by leveraging
weighted finite state transducers. Through experiments conducted on the
LibriSpeech and LibriVox datasets, we demonstrate that training ASR models with
OTC avoids performance degradation even with transcripts containing up to 70%
errors, a scenario where CTC models fail completely. Our implementation is
available at https://github.com/k2-fsa/icefall.