Incorporating L2 Phonemes Using Articulatory Features for Robust Speech
  Recognition
Jisung Wang Haram Lee Myungwoo Oh
The limited availability of non-native speech datasets presents a major
challenge in automatic speech recognition (ASR) to narrow the performance gap
between native and non-native speakers. To address this, the focus of this
study is on the efficient incorporation of the L2 phonemes, which in this work
refer to Korean phonemes, through articulatory feature analysis. This not only
enables accurate modeling of pronunciation variants but also allows for the
utilization of both native Korean and English speech datasets. We employ the
lattice-free maximum mutual information (LF-MMI) objective in an end-to-end
manner, to train the acoustic model to align and predict one of multiple
pronunciation candidates. Experimental results show that the proposed method
improves ASR accuracy for Korean L2 speech by training solely on L1 speech
data. Furthermore, fine-tuning on L2 speech improves recognition accuracy for
both L1 and L2 speech without performance trade-offs.