VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialogue
Yunshui Li Binyuan Hui Zhaochao Yin Wanwei He Run Luo Yuxing Long Min Yang Fei Huang Yongbin Li
Visually-grounded dialog systems, which integrate multiple modes of
communication such as text and visual inputs, have become an increasingly
popular area of investigation. However, the absence of a standardized
evaluation framework poses a challenge in assessing the development of this
field. To this end, we propose \textbf{VDialogUE}, a \textbf{V}isually-grounded
\textbf{Dialog}ue benchmark for \textbf{U}nified \textbf{E}valuation. It
defines five core multi-modal dialogue tasks and covers six datasets.
Furthermore, in order to provide a comprehensive assessment of the model's
performance across all tasks, we developed a novel evaluation metric called
VDscore, which is based on the Analytic Hierarchy Process~(AHP) method.
Additionally, we present a straightforward yet efficient baseline model, named
\textbf{VISIT}~(\textbf{VIS}ually-grounded d\textbf{I}alog
\textbf{T}ransformer), to promote the advancement of general multi-modal
dialogue systems. It progressively builds its multi-modal foundation and
dialogue capability via a two-stage pre-training strategy.
  We believe that the VDialogUE benchmark, along with the evaluation scripts
and our baseline models, will accelerate the development of visually-grounded
dialog systems and lead to the development of more sophisticated and effective
pre-trained models.