Zero shot VLMs for hate meme detection: Are we there yet?
Naquee Rizwan Paramananda Bhaskar Mithun Das Swadhin Satyaprakash Majhi Punyajoy Saha Animesh Mukherjee
Multimedia content on social media is rapidly evolving, with memes gaining
prominence as a distinctive form. Unfortunately, some malicious users exploit
memes to target individuals or vulnerable communities, making it imperative to
identify and address such instances of hateful memes. Extensive research has
been conducted to address this issue by developing hate meme detection models.
However, a notable limitation of traditional machine/deep learning models is
the requirement for labeled datasets for accurate classification. Recently, the
research community has witnessed the emergence of several visual language
models that have exhibited outstanding performance across various tasks. In
this study, we aim to investigate the efficacy of these visual language models
in handling intricate tasks such as hate meme detection. We use various prompt
settings to focus on zero-shot classification of hateful/harmful memes. Through
our analysis, we observe that large VLMs are still vulnerable for zero-shot
hate meme detection.