Explainable Multimodal Sentiment Analysis on Bengali Memes
Kazi Toufique Elahi Tasnuva Binte Rahman Shakil Shahriar Samir Sarker Sajib Kumar Saha Joy Faisal Muhammad Shah
Memes have become a distinctive and effective form of communication in the
digital era, attracting online communities and cutting across cultural
barriers. Even though memes are frequently linked with humor, they have an
amazing capacity to convey a wide range of emotions, including happiness,
sarcasm, frustration, and more. Understanding and interpreting the sentiment
underlying memes has become crucial in the age of information. Previous
research has explored text-based, image-based, and multimodal approaches,
leading to the development of models like CAPSAN and PromptHate for detecting
various meme categories. However, the study of low-resource languages like
Bengali memes remains scarce, with limited availability of publicly accessible
datasets. A recent contribution includes the introduction of the MemoSen
dataset. However, the achieved accuracy is notably low, and the dataset suffers
from imbalanced distribution. In this study, we employed a multimodal approach
using ResNet50 and BanglishBERT and achieved a satisfactory result of 0.71
weighted F1-score, performed comparison with unimodal approaches, and
interpreted behaviors of the models using explainable artificial intelligence
(XAI) techniques.