AesBench: An Expert Benchmark for Multimodal Large Language Models on
  Image Aesthetics Perception
Yipo Huang Quan Yuan Xiangfei Sheng Zhichao Yang Haoning Wu Pengfei Chen Yuzhe Yang Leida Li Weisi Lin
With collective endeavors, multimodal large language models (MLLMs) are
undergoing a flourishing development. However, their performances on image
aesthetics perception remain indeterminate, which is highly desired in
real-world applications. An obvious obstacle lies in the absence of a specific
benchmark to evaluate the effectiveness of MLLMs on aesthetic perception. This
blind groping may impede the further development of more advanced MLLMs with
aesthetic perception capacity. To address this dilemma, we propose AesBench, an
expert benchmark aiming to comprehensively evaluate the aesthetic perception
capacities of MLLMs through elaborate design across dual facets. (1) We
construct an Expert-labeled Aesthetics Perception Database (EAPD), which
features diversified image contents and high-quality annotations provided by
professional aesthetic experts. (2) We propose a set of integrative criteria to
measure the aesthetic perception abilities of MLLMs from four perspectives,
including Perception (AesP), Empathy (AesE), Assessment (AesA) and
Interpretation (AesI). Extensive experimental results underscore that the
current MLLMs only possess rudimentary aesthetic perception ability, and there
is still a significant gap between MLLMs and humans. We hope this work can
inspire the community to engage in deeper explorations on the aesthetic
potentials of MLLMs. Source data will be available at
https://github.com/yipoh/AesBench.