PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using
  Large Language Models
Nirmalendu Prakash Han Wang Nguyen Khoi Hoang Ming Shan Hee Roy Ka-Wei Lee
The proliferation of social media has given rise to a new form of
communication: memes. Memes are multimodal and often contain a combination of
text and visual elements that convey meaning, humor, and cultural significance.
While meme analysis has been an active area of research, little work has been
done on unsupervised multimodal topic modeling of memes, which is important for
content moderation, social media analysis, and cultural studies. We propose
\textsf{PromptMTopic}, a novel multimodal prompt-based model designed to learn
topics from both text and visual modalities by leveraging the language modeling
capabilities of large language models. Our model effectively extracts and
clusters topics learned from memes, considering the semantic interaction
between the text and visual modalities. We evaluate our proposed model through
extensive experiments on three real-world meme datasets, which demonstrate its
superiority over state-of-the-art topic modeling baselines in learning
descriptive topics in memes. Additionally, our qualitative analysis shows that
\textsf{PromptMTopic} can identify meaningful and culturally relevant topics
from memes. Our work contributes to the understanding of the topics and themes
of memes, a crucial form of communication in today's society.\\
\red{\textbf{Disclaimer: This paper contains sensitive content that may be
disturbing to some readers.}}