Improving RNN-Transducers with Acoustic LookAhead
Vinit S. Unni Ashish Mittal Preethi Jyothi Sunita Sarawagi
RNN-Transducers (RNN-Ts) have gained widespread acceptance as an end-to-end
model for speech to text conversion because of their high accuracy and
streaming capabilities. A typical RNN-T independently encodes the input audio
and the text context, and combines the two encodings by a thin joint network.
While this architecture provides SOTA streaming accuracy, it also makes the
model vulnerable to strong LM biasing which manifests as multi-step
hallucination of text without acoustic evidence. In this paper we propose
LookAhead that makes text representations more acoustically grounded by looking
ahead into the future within the audio input. This technique yields a
significant 5%-20% relative reduction in word error rate on both in-domain and
out-of-domain evaluation sets.