Multi-Branch Network for Imagery Emotion Prediction
Quoc-Bao Ninh Hai-Chan Nguyen Triet Huynh Trung-Nghia Le
For a long time, images have proved perfect at both storing and conveying
rich semantics, especially human emotions. A lot of research has been conducted
to provide machines with the ability to recognize emotions in photos of people.
Previous methods mostly focus on facial expressions but fail to consider the
scene context, meanwhile scene context plays an important role in predicting
emotions, leading to more accurate results. In addition,
Valence-Arousal-Dominance (VAD) values offer a more precise quantitative
understanding of continuous emotions, yet there has been less emphasis on
predicting them compared to discrete emotional categories. In this paper, we
present a novel Multi-Branch Network (MBN), which utilizes various source
information, including faces, bodies, and scene contexts to predict both
discrete and continuous emotions in an image. Experimental results on EMOTIC
dataset, which contains large-scale images of people in unconstrained
situations labeled with 26 discrete categories of emotions and VAD values, show
that our proposed method significantly outperforms state-of-the-art methods
with 28.4% in mAP and 0.93 in MAE. The results highlight the importance of
utilizing multiple contextual information in emotion prediction and illustrate
the potential of our proposed method in a wide range of applications, such as
effective computing, human-computer interaction, and social robotics. Source
code:
https://github.com/BaoNinh2808/Multi-Branch-Network-for-Imagery-Emotion-Prediction