Predicting positive transfer for improved low-resource speech
  recognition using acoustic pseudo-tokens
Nay San Georgios Paraskevopoulos Aryaman Arora Xiluo He Prabhjot Kaur Oliver Adams Dan Jurafsky
While massively multilingual speech models like wav2vec 2.0 XLSR-128 can be
directly fine-tuned for automatic speech recognition (ASR), downstream
performance can still be relatively poor on languages that are
under-represented in the pre-training data. Continued pre-training on 70-200
hours of untranscribed speech in these languages can help -- but what about
languages without that much recorded data? For such cases, we show that
supplementing the target language with data from a similar, higher-resource
'donor' language can help. For example, continued pre-training on only 10 hours
of low-resource Punjabi supplemented with 60 hours of donor Hindi is almost as
good as continued pretraining on 70 hours of Punjabi. By contrast, sourcing
data from less similar donors like Bengali does not improve ASR performance. To
inform donor language selection, we propose a novel similarity metric based on
the sequence distribution of induced acoustic units: the Acoustic Token
Distribution Similarity (ATDS). Across a set of typologically different target
languages (Punjabi, Galician, Iban, Setswana), we show that the ATDS between
the target language and its candidate donors precisely predicts target language
ASR performance.