Speech emotion recognition from voice messages recorded in the wild
Lucía Gómez-Zaragozá Óscar Valls Rocío del Amor María José Castro-Bleda Valery Naranjo Mariano Alcañiz Raya Javier Marín-Morales
Emotion datasets used for Speech Emotion Recognition (SER) often contain
acted or elicited speech, limiting their applicability in real-world scenarios.
In this work, we used the Emotional Voice Messages (EMOVOME) database,
including spontaneous voice messages from conversations of 100 Spanish speakers
on a messaging app, labeled in continuous and discrete emotions by expert and
non-expert annotators. We created speaker-independent SER models using the
eGeMAPS features, transformer-based models and their combination. We compared
the results with reference databases and analyzed the influence of annotators
and gender fairness. The pre-trained Unispeech-L model and its combination with
eGeMAPS achieved the highest results, with 61.64% and 55.57% Unweighted
Accuracy (UA) for 3-class valence and arousal prediction respectively, a 10%
improvement over baseline models. For the emotion categories, 42.58% UA was
obtained. EMOVOME performed lower than the acted RAVDESS database. The elicited
IEMOCAP database also outperformed EMOVOME in the prediction of emotion
categories, while similar results were obtained in valence and arousal.
Additionally, EMOVOME outcomes varied with annotator labels, showing superior
results and better fairness when combining expert and non-expert annotations.
This study significantly contributes to the evaluation of SER models in
real-life situations, advancing in the development of applications for
analyzing spontaneous voice messages.