Fast Word Error Rate Estimation Using Self-Supervised Representations
  For Speech And Text
Chanho Park Chengsong Lu Mingjie Chen Thomas Hain
The quality of automatic speech recognition (ASR) is typically measured by
word error rate (WER). WER estimation is a task aiming to predict the WER of an
ASR system, given a speech utterance and a transcription. This task has gained
increasing attention while advanced ASR systems are trained on large amounts of
data. In this case, WER estimation becomes necessary in many scenarios, for
example, selecting training data with unknown transcription quality or
estimating the testing performance of an ASR system without ground truth
transcriptions. Facing large amounts of data, the computation efficiency of a
WER estimator becomes essential in practical applications. However, previous
works usually did not consider it as a priority. In this paper, a Fast WER
estimator (Fe-WER) using self-supervised learning representation (SSLR) is
introduced. The estimator is built upon SSLR aggregated by average pooling. The
results show that Fe-WER outperformed the e-WER3 baseline relatively by 19.69%
and 7.16% on Ted-Lium3 in both evaluation metrics of root mean square error and
Pearson correlation coefficient, respectively. Moreover, the estimation
weighted by duration was 10.43% when the target was 10.88%. Lastly, the
inference speed was about 4x in terms of a real-time factor.