Multiscale Contextual Learning for Speech Emotion Recognition in
  Emergency Call Center Conversations
Théo Deschamps-Berger Lori Lamel Laurence Devillers
Emotion recognition in conversations is essential for ensuring advanced
human-machine interactions. However, creating robust and accurate emotion
recognition systems in real life is challenging, mainly due to the scarcity of
emotion datasets collected in the wild and the inability to take into account
the dialogue context. The CEMO dataset, composed of conversations between
agents and patients during emergency calls to a French call center, fills this
gap. The nature of these interactions highlights the role of the emotional flow
of the conversation in predicting patient emotions, as context can often make a
difference in understanding actual feelings. This paper presents a multi-scale
conversational context learning approach for speech emotion recognition, which
takes advantage of this hypothesis. We investigated this approach on both
speech transcriptions and acoustic segments. Experimentally, our method uses
the previous or next information of the targeted segment. In the text domain,
we tested the context window using a wide range of tokens (from 10 to 100) and
at the speech turns level, considering inputs from both the same and opposing
speakers. According to our tests, the context derived from previous tokens has
a more significant influence on accurate prediction than the following tokens.
Furthermore, taking the last speech turn of the same speaker in the
conversation seems useful. In the acoustic domain, we conducted an in-depth
analysis of the impact of the surrounding emotions on the prediction. While
multi-scale conversational context learning using Transformers can enhance
performance in the textual modality for emergency call recordings,
incorporating acoustic context is more challenging.