Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech
  Detection
Atanu Mandal Gargi Roy Amit Barman Indranil Dutta Sudip Kumar Naskar
With the recent surge and exponential growth of social media usage,
scrutinizing social media content for the presence of any hateful content is of
utmost importance. Researchers have been diligently working since the past
decade on distinguishing between content that promotes hatred and content that
does not. Traditionally, the main focus has been on analyzing textual content.
However, recent research attempts have also commenced into the identification
of audio-based content. Nevertheless, studies have shown that relying solely on
audio or text-based content may be ineffective, as recent upsurge indicates
that individuals often employ sarcasm in their speech and writing. To overcome
these challenges, we present an approach to identify whether a speech promotes
hate or not utilizing both audio and textual representations. Our methodology
is based on the Transformer framework that incorporates both audio and text
sampling, accompanied by our very own layer called "Attentive Fusion". The
results of our study surpassed previous state-of-the-art techniques, achieving
an impressive macro F1 score of 0.927 on the Test Set.