Direct Punjabi to English speech translation using discrete units
Prabhjot Kaur L. Andrew M. Bush Weisong Shi
Speech-to-speech translation is yet to reach the same level of coverage as
text-to-text translation systems. The current speech technology is highly
limited in its coverage of over 7000 languages spoken worldwide, leaving more
than half of the population deprived of such technology and shared experiences.
With voice-assisted technology (such as social robots and speech-to-text apps)
and auditory content (such as podcasts and lectures) on the rise, ensuring that
the technology is available for all is more important than ever. Speech
translation can play a vital role in mitigating technological disparity and
creating a more inclusive society. With a motive to contribute towards speech
translation research for low-resource languages, our work presents a direct
speech-to-speech translation model for one of the Indic languages called
Punjabi to English. Additionally, we explore the performance of using a
discrete representation of speech called discrete acoustic units as input to
the Transformer-based translation model. The model, abbreviated as Unit-to-Unit
Translation (U2UT), takes a sequence of discrete units of the source language
(the language being translated from) and outputs a sequence of discrete units
of the target language (the language being translated to). Our results show
that the U2UT model performs better than the Speech-to-Unit Translation (S2UT)
model by a 3.69 BLEU score.