Benchmarking Sequential Visual Input Reasoning and Prediction in
  Multimodal Large Language Models
Mingwei Zhu Leigang Sha Yu Shu Kangjia Zhao Tiancheng Zhao Jianwei Yin
Multimodal large language models (MLLMs) have shown great potential in
perception and interpretation tasks, but their capabilities in predictive
reasoning remain under-explored. To address this gap, we introduce a novel
benchmark that assesses the predictive reasoning capabilities of MLLMs across
diverse scenarios. Our benchmark targets three important domains: abstract
pattern reasoning, human activity prediction, and physical interaction
prediction. We further develop three evaluation methods powered by large
language model to robustly quantify a model's performance in predicting and
reasoning the future based on multi-visual context. Empirical experiments
confirm the soundness of the proposed benchmark and evaluation methods via
rigorous testing and reveal pros and cons of current popular MLLMs in the task
of predictive reasoning. Lastly, our proposed benchmark provides a standardized
evaluation framework for MLLMs and can facilitate the development of more
advanced models that can reason and predict over complex long sequence of
multimodal input.