Transcending Traditional Boundaries: Leveraging Inter-Annotator
  Agreement (IAA) for Enhancing Data Management Operations (DMOps)
Damrin Kim NamHyeok Kim Chanjun Park Harksoo Kim
This paper presents a novel approach of leveraging Inter-Annotator Agreement
(IAA), traditionally used for assessing labeling consistency, to optimize Data
Management Operations (DMOps). We advocate for the use of IAA in predicting the
labeling quality of individual annotators, leading to cost and time efficiency
in data production. Additionally, our work highlights the potential of IAA in
forecasting document difficulty, thereby boosting the data construction
process's overall efficiency. This research underscores IAA's broader
application potential in data-driven research optimization and holds
significant implications for large-scale data projects prioritizing efficiency,
cost reduction, and high-quality data.