Hystoc: Obtaining word confidences for fusion of end-to-end ASR systems
Karel Beneš Martin Kocour Lukáš Burget
End-to-end (e2e) systems have recently gained wide popularity in automatic
speech recognition. However, these systems do generally not provide
well-calibrated word-level confidences. In this paper, we propose Hystoc, a
simple method for obtaining word-level confidences from hypothesis-level
scores. Hystoc is an iterative alignment procedure which turns hypotheses from
an n-best output of the ASR system into a confusion network. Eventually,
word-level confidences are obtained as posterior probabilities in the
individual bins of the confusion network. We show that Hystoc provides
confidences that correlate well with the accuracy of the ASR hypothesis.
Furthermore, we show that utilizing Hystoc in fusion of multiple e2e ASR
systems increases the gains from the fusion by up to 1\,\% WER absolute on
Spanish RTVE2020 dataset. Finally, we experiment with using Hystoc for direct
fusion of n-best outputs from multiple systems, but we only achieve minor gains
when fusing very similar systems.