SUN Team's Contribution to ABAW 2024 Competition: Audio-visual
  Valence-Arousal Estimation and Expression Recognition
Denis Dresvyanskiy Maxim Markitantov Jiawei Yu Peitong Li Heysem Kaya Alexey Karpov
As emotions play a central role in human communication, automatic emotion
recognition has attracted increasing attention in the last two decades. While
multimodal systems enjoy high performances on lab-controlled data, they are
still far from providing ecological validity on non-lab-controlled, namely
'in-the-wild' data. This work investigates audiovisual deep learning approaches
for emotion recognition in-the-wild problem. We particularly explore the
effectiveness of architectures based on fine-tuned Convolutional Neural
Networks (CNN) and Public Dimensional Emotion Model (PDEM), for video and audio
modality, respectively. We compare alternative temporal modeling and fusion
strategies using the embeddings from these multi-stage trained
modality-specific Deep Neural Networks (DNN). We report results on the AffWild2
dataset under Affective Behavior Analysis in-the-Wild 2024 (ABAW'24) challenge
protocol.