How to Evaluate Coreference in Literary Texts?
Ana-Isabel Duron-Tejedor Pascal Amsili Thierry Poibeau
In this short paper, we examine the main metrics used to evaluate textual
coreference and we detail some of their limitations. We show that a unique
score cannot represent the full complexity of the problem at stake, and is thus
uninformative, or even misleading. We propose a new way of evaluating
coreference, taking into account the context (in our case, the analysis of
fictions, esp. novels). More specifically, we propose to distinguish long
coreference chains (corresponding to main characters), from short ones
(corresponding to secondary characters), and singletons (isolated elements).
This way, we hope to get more interpretable and thus more informative results
through evaluation.