REFinD: Relation Extraction Financial Dataset
Simerjot Kaur Charese Smiley Akshat Gupta Joy Sain Dongsheng Wang Suchetha Siddagangappa Toyin Aguda Sameena Shah
A number of datasets for Relation Extraction (RE) have been created to aide
downstream tasks such as information retrieval, semantic search, question
answering and textual entailment. However, these datasets fail to capture
financial-domain specific challenges since most of these datasets are compiled
using general knowledge sources such as Wikipedia, web-based text and news
articles, hindering real-life progress and adoption within the financial world.
To address this limitation, we propose REFinD, the first large-scale annotated
dataset of relations, with $\sim$29K instances and 22 relations amongst 8 types
of entity pairs, generated entirely over financial documents. We also provide
an empirical evaluation with various state-of-the-art models as benchmarks for
the RE task and highlight the challenges posed by our dataset. We observed that
various state-of-the-art deep learning models struggle with numeric inference,
relational and directional ambiguity.