Mispronunciation detection using self-supervised speech representations
Jazmin Vidal Pablo Riera Luciana Ferrer
In recent years, self-supervised learning (SSL) models have produced
promising results in a variety of speech-processing tasks, especially in
contexts of data scarcity. In this paper, we study the use of SSL models for
the task of mispronunciation detection for second language learners. We compare
two downstream approaches: 1) training the model for phone recognition (PR)
using native English data, and 2) training a model directly for the target task
using non-native English data. We compare the performance of these two
approaches for various SSL representations as well as a representation
extracted from a traditional DNN-based speech recognition model. We evaluate
the models on L2Arctic and EpaDB, two datasets of non-native speech annotated
with pronunciation labels at the phone level. Overall, we find that using a
downstream model trained for the target task gives the best performance and
that most upstream models perform similarly for the task.