Improved Long-Form Speech Recognition by Jointly Modeling the Primary
  and Non-primary Speakers
Guru Prakash Arumugam Shuo-yiin Chang Tara N. Sainath Rohit Prabhavalkar Quan Wang Shaan Bijwadia
ASR models often suffer from a long-form deletion problem where the model
predicts sequential blanks instead of words when transcribing a lengthy audio
(in the order of minutes or hours). From the perspective of a user or
downstream system consuming the ASR results, this behavior can be perceived as
the model "being stuck", and potentially make the product hard to use. One of
the culprits for long-form deletion is training-test data mismatch, which can
happen even when the model is trained on diverse and large-scale data collected
from multiple application domains. In this work, we introduce a novel technique
to simultaneously model different groups of speakers in the audio along with
the standard transcript tokens. Speakers are grouped as primary and
non-primary, which connects the application domains and significantly
alleviates the long-form deletion problem. This improved model neither needs
any additional training data nor incurs additional training or inference cost.