Learning Cross-lingual Mappings for Data Augmentation to Improve
  Low-Resource Speech Recognition
Muhammad Umar Farooq Thomas Hain
Exploiting cross-lingual resources is an effective way to compensate for data
scarcity of low resource languages. Recently, a novel multilingual model fusion
technique has been proposed where a model is trained to learn cross-lingual
acoustic-phonetic similarities as a mapping function. However, handcrafted
lexicons have been used to train hybrid DNN-HMM ASR systems. To remove this
dependency, we extend the concept of learnable cross-lingual mappings for
end-to-end speech recognition. Furthermore, mapping models are employed to
transliterate the source languages to the target language without using
parallel data. Finally, the source audio and its transliteration is used for
data augmentation to retrain the target language ASR. The results show that any
source language ASR model can be used for a low-resource target language
recognition followed by proposed mapping model. Furthermore, data augmentation
results in a relative gain up to 5% over baseline monolingual model.