The (R)Evolution of Multimodal Large Language Models: A Survey
Davide Caffagni Federico Cocchi Luca Barsellotti Nicholas Moratelli Sara Sarto Lorenzo Baraldi Lorenzo Baraldi Marcella Cornia Rita Cucchiara
Connecting text and visual modalities plays an essential role in generative
intelligence. For this reason, inspired by the success of large language
models, significant research efforts are being devoted to the development of
Multimodal Large Language Models (MLLMs). These models can seamlessly integrate
visual and textual modalities, both as input and output, while providing a
dialogue-based interface and instruction-following capabilities. In this paper,
we provide a comprehensive review of recent visual-based MLLMs, analyzing their
architectural choices, multimodal alignment strategies, and training
techniques. We also conduct a detailed analysis of these models across a wide
range of tasks, including visual grounding, image generation and editing,
visual understanding, and domain-specific applications. Additionally, we
compile and describe training datasets and evaluation benchmarks, conducting
comparisons among existing models in terms of performance and computational
requirements. Overall, this survey offers a comprehensive overview of the
current state of the art, laying the groundwork for future MLLMs.