Improving Speech Recognition for African American English With Audio
  Classification
Shefali Garg Zhouyuan Huo Khe Chai Sim Suzan Schwartz Mason Chua Alëna Aksënova Tsendsuren Munkhdalai Levi King Darryl Wright Zion Mengesha Dongseong Hwang Tara Sainath Françoise Beaufays Pedro Moreno Mengibar
Automatic speech recognition (ASR) systems have been shown to have large
quality disparities between the language varieties they are intended or
expected to recognize. One way to mitigate this is to train or fine-tune models
with more representative datasets. But this approach can be hindered by limited
in-domain data for training and evaluation. We propose a new way to improve the
robustness of a US English short-form speech recognizer using a small amount of
out-of-domain (long-form) African American English (AAE) data. We use CORAAL,
YouTube and Mozilla Common Voice to train an audio classifier to approximately
output whether an utterance is AAE or some other variety including Mainstream
American English (MAE). By combining the classifier output with coarse
geographic information, we can select a subset of utterances from a large
corpus of untranscribed short-form queries for semi-supervised learning at
scale. Fine-tuning on this data results in a 38.5% relative word error rate
disparity reduction between AAE and MAE without reducing MAE quality.