How Paralingual are Paralinguistic Representations? A Case Study in
  Speech Emotion Recognition
Orchid Chetia Phukan Gautam Siddharth Kashyap Arun Balaji Buduru Rajesh Sharma
Pre-trained Models (PTMs) have facilitated substantial progress in the field
of Speech Emotion Recognition (SER). SER is an area with applications ranging
from HumanComputer Interaction to Healthcare. Recent studies have leveraged
various PTM representations as input features for downstream models for SER.
PTM specifically pre-trained for paralinguistic tasks have obtained
state-of-the-art (SOTA) performance for SER. However, such PTM haven't been
evaluated for SER in multilingual settings and experimented only with English.
So, we fill this gap, by performing a comprehensive comparative study of five
PTMs (TRILLsson, wav2vec2, XLS-R, x-vector, Whisper) for assessing the
effectiveness of paralingual PTM (TRILLsson) for SER across multiple languages.
Representations from TRILLsson achieved the best performance among all the
PTMs. This demonstrates that TRILLsson is able to effectively capture the
various paralinguistic features from speech data for better SER. We also show
that downstream models using TRILLsson representations achieve SOTA performance
in terms of accuracy across various multi-lingual datasets.