Gujarati-English Code-Switching Speech Recognition using ensemble
  prediction of spoken language
Yash Sharma Basil Abraham Preethi Jyothi
An important and difficult task in code-switched speech recognition is to
recognize the language, as lots of words in two languages can sound similar,
especially in some accents. We focus on improving performance of end-to-end
Automatic Speech Recognition models by conditioning transformer layers on
language ID of words and character in the output in an per layer supervised
manner. To this end, we propose two methods of introducing language specific
parameters and explainability in the multi-head attention mechanism, and
implement a Temporal Loss that helps maintain continuity in input alignment.
Despite being unable to reduce WER significantly, our method shows promise in
predicting the correct language from just spoken data. We introduce
regularization in the language prediction by dropping LID in the sequence,
which helps align long repeated output sequences.