GROOViST: A Metric for Grounding Objects in Visual Storytelling
Aditya K Surikuchi Sandro Pezzelle Raquel Fernández
A proper evaluation of stories generated for a sequence of images -- the task
commonly referred to as visual storytelling -- must consider multiple aspects,
such as coherence, grammatical correctness, and visual grounding. In this work,
we focus on evaluating the degree of grounding, that is, the extent to which a
story is about the entities shown in the images. We analyze current metrics,
both designed for this purpose and for general vision-text alignment. Given
their observed shortcomings, we propose a novel evaluation tool, GROOViST, that
accounts for cross-modal dependencies, temporal misalignments (the fact that
the order in which entities appear in the story and the image sequence may not
match), and human intuitions on visual grounding. An additional advantage of
GROOViST is its modular design, where the contribution of each component can be
assessed and interpreted individually.