Neural Multimodal Topic Modeling: A Comprehensive Evaluation
Felipe González-Pizarro Giuseppe Carenini
Neural topic models can successfully find coherent and diverse topics in
textual data. However, they are limited in dealing with multimodal datasets
(e.g., images and text). This paper presents the first systematic and
comprehensive evaluation of multimodal topic modeling of documents containing
both text and images. In the process, we propose two novel topic modeling
solutions and two novel evaluation metrics. Overall, our evaluation on an
unprecedented rich and diverse collection of datasets indicates that both of
our models generate coherent and diverse topics. Nevertheless, the extent to
which one method outperforms the other depends on the metrics and dataset
combinations, which suggests further exploration of hybrid solutions in the
future. Notably, our succinct human evaluation aligns with the outcomes
determined by our proposed metrics. This alignment not only reinforces the
credibility of our metrics but also highlights the potential for their
application in guiding future multimodal topic modeling endeavors.