On Detecting Cherry-picking in News Coverage Using Large Language Models
Israa Jaradat Haiqi Zhang Chengkai Li
Cherry-picking refers to the deliberate selection of evidence or facts that
favor a particular viewpoint while ignoring or distorting evidence that
supports an opposing perspective. Manually identifying instances of
cherry-picked statements in news stories can be challenging, particularly when
the opposing viewpoint's story is absent. This study introduces Cherry, an
innovative approach for automatically detecting cherry-picked statements in
news articles by finding missing important statements in the target news story.
Cherry utilizes the analysis of news coverage from multiple sources to identify
instances of cherry-picking. Our approach relies on language models that
consider contextual information from other news sources to classify statements
based on their importance to the event covered in the target news story.
Furthermore, this research introduces a novel dataset specifically designed for
cherry-picking detection, which was used to train and evaluate the performance
of the models. Our best performing model achieves an F-1 score of about %89 in
detecting important statements when tested on unseen set of news stories.
Moreover, results show the importance incorporating external knowledge from
alternative unbiased narratives when assessing a statement's importance.