Fusion approaches for emotion recognition from speech using acoustic and
  text-based features
Leonardo Pepino Pablo Riera Luciana Ferrer Agustin Gravano
In this paper, we study different approaches for classifying emotions from
speech using acoustic and text-based features. We propose to obtain
contextualized word embeddings with BERT to represent the information contained
in speech transcriptions and show that this results in better performance than
using Glove embeddings. We also propose and compare different strategies to
combine the audio and text modalities, evaluating them on IEMOCAP and
MSP-PODCAST datasets. We find that fusing acoustic and text-based systems is
beneficial on both datasets, though only subtle differences are observed across
the evaluated fusion approaches. Finally, for IEMOCAP, we show the large effect
that the criteria used to define the cross-validation folds have on results. In
particular, the standard way of creating folds for this dataset results in a
highly optimistic estimation of performance for the text-based system,
suggesting that some previous works may overestimate the advantage of
incorporating transcriptions.