Transcription and translation of videos using fine-tuned XLSR Wav2Vec2
  on custom dataset and mBART
Aniket Tathe Anand Kamble Suyash Kumbharkar Atharva Bhandare Anirban C. Mitra
This research addresses the challenge of training an ASR model for
personalized voices with minimal data. Utilizing just 14 minutes of custom
audio from a YouTube video, we employ Retrieval-Based Voice Conversion (RVC) to
create a custom Common Voice 16.0 corpus. Subsequently, a Cross-lingual
Self-supervised Representations (XLSR) Wav2Vec2 model is fine-tuned on this
dataset. The developed web-based GUI efficiently transcribes and translates
input Hindi videos. By integrating XLSR Wav2Vec2 and mBART, the system aligns
the translated text with the video timeline, delivering an accessible solution
for multilingual video content transcription and translation for personalized
voice.