Information Extraction in Domain and Generic Documents: Findings from
  Heuristic-based and Data-driven Approaches
Shiyu Yuan Carlo Lipizzi
Information extraction (IE) plays very important role in natural language
processing (NLP) and is fundamental to many NLP applications that used to
extract structured information from unstructured text data. Heuristic-based
searching and data-driven learning are two main stream implementation
approaches. However, no much attention has been paid to document genre and
length influence on IE tasks. To fill the gap, in this study, we investigated
the accuracy and generalization abilities of heuristic-based searching and
data-driven to perform two IE tasks: named entity recognition (NER) and
semantic role labeling (SRL) on domain-specific and generic documents with
different length. We posited two hypotheses: first, short documents may yield
better accuracy results compared to long documents; second, generic documents
may exhibit superior extraction outcomes relative to domain-dependent documents
due to training document genre limitations. Our findings reveals that no single
method demonstrated overwhelming performance in both tasks. For named entity
extraction, data-driven approaches outperformed symbolic methods in terms of
accuracy, particularly in short texts. In the case of semantic roles
extraction, we observed that heuristic-based searching method and data-driven
based model with syntax representation surpassed the performance of pure
data-driven approach which only consider semantic information. Additionally, we
discovered that different semantic roles exhibited varying accuracy levels with
the same method. This study offers valuable insights for downstream text mining
tasks, such as NER and SRL, when addressing various document features and
genres.