Evaluating Dynamic Topic Models
Charu James Mayank Nagda Nooshin Haji Ghassemi Marius Kloft Sophie Fellenz
There is a lack of quantitative measures to evaluate the progression of
topics through time in dynamic topic models (DTMs). Filling this gap, we
propose a novel evaluation measure for DTMs that analyzes the changes in the
quality of each topic over time. Additionally, we propose an extension
combining topic quality with the model's temporal consistency. We demonstrate
the utility of the proposed measure by applying it to synthetic data and data
from existing DTMs. We also conducted a human evaluation, which indicates that
the proposed measure correlates well with human judgment. Our findings may help
in identifying changing topics, evaluating different DTMs, and guiding future
research in this area.