Unsupervised Pre-Training for Vietnamese Automatic Speech Recognition in
  the HYKIST Project
Khai Le-Duc
In today's interconnected globe, moving abroad is more and more prevalent,
whether it's for employment, refugee resettlement, or other causes. Language
difficulties between natives and immigrants present a common issue on a daily
basis, especially in medical domain. This can make it difficult for patients
and doctors to communicate during anamnesis or in the emergency room, which
compromises patient care. The goal of the HYKIST Project is to develop a speech
translation system to support patient-doctor communication with ASR and MT.
  ASR systems have recently displayed astounding performance on particular
tasks for which enough quantities of training data are available, such as
LibriSpeech. Building a good model is still difficult due to a variety of
speaking styles, acoustic and recording settings, and a lack of in-domain
training data. In this thesis, we describe our efforts to construct ASR systems
for a conversational telephone speech recognition task in the medical domain
for Vietnamese language to assist emergency room contact between doctors and
patients across linguistic barriers. In order to enhance the system's
performance, we investigate various training schedules and data combining
strategies. We also examine how best to make use of the little data that is
available. The use of publicly accessible models like XLSR-53 is compared to
the use of customized pre-trained models, and both supervised and unsupervised
approaches are utilized using wav2vec 2.0 as architecture.