Noise-Robust Keyword Spotting through Self-supervised Pretraining
Jacob Mørk Holger Severin Bovbjerg Gergely Kiss Zheng-Hua Tan
Voice assistants are now widely available, and to activate them a keyword
spotting (KWS) algorithm is used. Modern KWS systems are mainly trained using
supervised learning methods and require a large amount of labelled data to
achieve a good performance. Leveraging unlabelled data through self-supervised
learning (SSL) has been shown to increase the accuracy in clean conditions.
This paper explores how SSL pretraining such as Data2Vec can be used to enhance
the robustness of KWS models in noisy conditions, which is under-explored.
  Models of three different sizes are pretrained using different pretraining
approaches and then fine-tuned for KWS. These models are then tested and
compared to models trained using two baseline supervised learning methods, one
being standard training using clean data and the other one being multi-style
training (MTR). The results show that pretraining and fine-tuning on clean data
is superior to supervised learning on clean data across all testing conditions,
and superior to supervised MTR for testing conditions of SNR above 5 dB. This
indicates that pretraining alone can increase the model's robustness. Finally,
it is found that using noisy data for pretraining models, especially with the
Data2Vec-denoising approach, significantly enhances the robustness of KWS
models in noisy conditions.