Capturing Spectral and Long-term Contextual Information for Speech
  Emotion Recognition Using Deep Learning Techniques
Samiul Islam Md. Maksudul Haque Abu Jobayer Md. Sadat
Traditional approaches in speech emotion recognition, such as LSTM, CNN, RNN,
SVM, and MLP, have limitations such as difficulty capturing long-term
dependencies in sequential data, capturing the temporal dynamics, and
struggling to capture complex patterns and relationships in multimodal data.
This research addresses these shortcomings by proposing an ensemble model that
combines Graph Convolutional Networks (GCN) for processing textual data and the
HuBERT transformer for analyzing audio signals. We found that GCNs excel at
capturing Long-term contextual dependencies and relationships within textual
data by leveraging graph-based representations of text and thus detecting the
contextual meaning and semantic relationships between words. On the other hand,
HuBERT utilizes self-attention mechanisms to capture long-range dependencies,
enabling the modeling of temporal dynamics present in speech and capturing
subtle nuances and variations that contribute to emotion recognition. By
combining GCN and HuBERT, our ensemble model can leverage the strengths of both
approaches. This allows for the simultaneous analysis of multimodal data, and
the fusion of these modalities enables the extraction of complementary
information, enhancing the discriminative power of the emotion recognition
system. The results indicate that the combined model can overcome the
limitations of traditional methods, leading to enhanced accuracy in recognizing
emotions from speech.