MEGAnno+: A Human-LLM Collaborative Annotation System
Hannah Kim Kushan Mitra Rafael Li Chen Sajjadur Rahman Dan Zhang
Large language models (LLMs) can label data faster and cheaper than humans
for various NLP tasks. Despite their prowess, LLMs may fall short in
understanding of complex, sociocultural, or domain-specific context,
potentially leading to incorrect annotations. Therefore, we advocate a
collaborative approach where humans and LLMs work together to produce reliable
and high-quality labels. We present MEGAnno+, a human-LLM collaborative
annotation system that offers effective LLM agent and annotation management,
convenient and robust LLM annotation, and exploratory verification of LLM
labels by humans.