ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for
  Complicated Chart Reasoning
Renqiu Xia Bo Zhang Hancheng Ye Xiangchao Yan Qi Liu Hongbin Zhou Zijun Chen Min Dou Botian Shi Junchi Yan Yu Qiao
Recently, many versatile Multi-modal Large Language Models (MLLMs) have
emerged continuously. However, their capacity to query information depicted in
visual charts and engage in reasoning based on the queried contents remains
under-explored. In this paper, to comprehensively and rigorously benchmark the
ability of the off-the-shelf MLLMs in the chart domain, we construct ChartX, a
multi-modal evaluation set covering 18 chart types, 7 chart tasks, 22
disciplinary topics, and high-quality chart data. Besides, we develop ChartVLM
to offer a new perspective on handling multi-modal tasks that strongly depend
on interpretable patterns, such as reasoning tasks in the field of charts or
geometric images. We evaluate the chart-related ability of mainstream MLLMs and
our ChartVLM on the proposed ChartX evaluation set. Extensive experiments
demonstrate that ChartVLM surpasses both versatile and chart-related large
models, achieving results comparable to GPT-4V. We believe that our study can
pave the way for further exploration in creating a more comprehensive chart
evaluation set and developing more interpretable multi-modal models. Both
ChartX and ChartVLM are available at:
https://github.com/UniModal4Reasoning/ChartVLM