Feature Normalization for Fine-tuning Self-Supervised Models in Speech
  Enhancement
Hejung Yang Hong-Goo Kang
Large, pre-trained representation models trained using self-supervised
learning have gained popularity in various fields of machine learning because
they are able to extract high-quality salient features from input data. As
such, they have been frequently used as base networks for various pattern
classification tasks such as speech recognition. However, not much research has
been conducted on applying these types of models to the field of speech signal
generation. In this paper, we investigate the feasibility of using pre-trained
speech representation models for a downstream speech enhancement task. To
alleviate mismatches between the input features of the pre-trained model and
the target enhancement model, we adopt a novel feature normalization technique
to smoothly link these modules together. Our proposed method enables
significant improvements in speech quality compared to baselines when combined
with various types of pre-trained speech models.