M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets
Gaurish Thakkar Sherzod Hakimov Marko Tadić
In recent years, multimodal natural language processing, aimed at learning
from diverse data types, has garnered significant attention. However, there
needs to be more clarity when it comes to analysing multimodal tasks in
multi-lingual contexts. While prior studies on sentiment analysis of tweets
have predominantly focused on the English language, this paper addresses this
gap by transforming an existing textual Twitter sentiment dataset into a
multimodal format through a straightforward curation process. Our work opens up
new avenues for sentiment-related research within the research community.
Additionally, we conduct baseline experiments utilising this augmented dataset
and report the findings. Notably, our evaluations reveal that when comparing
unimodal and multimodal configurations, using a sentiment-tuned large language
model as a text encoder performs exceptionally well.