Globally Normalising the Transducer for Streaming Speech Recognition
Rogier van Dalen
The Transducer (e.g. RNN-Transducer or Conformer-Transducer) generates an
output label sequence as it traverses the input sequence. It is straightforward
to use in streaming mode, where it generates partial hypotheses before the
complete input has been seen. This makes it popular in speech recognition.
However, in streaming mode the Transducer has a mathematical flaw which, simply
put, restricts the model's ability to change its mind. The fix is to replace
local normalisation (e.g. a softmax) with global normalisation, but then the
loss function becomes impossible to evaluate exactly. A recent paper proposes
to solve this by approximating the model, severely degrading performance.
Instead, this paper proposes to approximate the loss function, allowing global
normalisation to apply to a state-of-the-art streaming model. Global
normalisation reduces its word error rate by 9-11% relative, closing almost
half the gap between streaming and lookahead mode.