What Can an Accent Identifier Learn? Probing Phonetic and Prosodic
  Information in a Wav2vec2-based Accent Identification Model
Mu Yang Ram C. M. C. Shekar Okim Kang John H. L. Hansen
This study is focused on understanding and quantifying the change in phoneme
and prosody information encoded in the Self-Supervised Learning (SSL) model,
brought by an accent identification (AID) fine-tuning task. This problem is
addressed based on model probing. Specifically, we conduct a systematic
layer-wise analysis of the representations of the Transformer layers on a
phoneme correlation task, and a novel word-level prosody prediction task. We
compare the probing performance of the pre-trained and fine-tuned SSL models.
Results show that the AID fine-tuning task steers the top 2 layers to learn
richer phoneme and prosody representation. These changes share some
similarities with the effects of fine-tuning with an Automatic Speech
Recognition task. In addition, we observe strong accent-specific phoneme
representations in layer 9. To sum up, this study provides insights into the
understanding of SSL features and their interactions with fine-tuning tasks.