EASE: An Easily-Customized Annotation System Powered by Efficiency
  Enhancement Mechanisms
Naihao Deng Yikai Liu Mingye Chen Winston Wu Siyang Liu Yulong Chen Yue Zhang Rada Mihalcea
The performance of current supervised AI systems is tightly connected to the
availability of annotated datasets. Annotations are usually collected through
annotation tools, which are often designed for specific tasks and are difficult
to customize. Moreover, existing annotation tools with an active learning
mechanism often only support limited use cases. To address these limitations,
we present EASE, an Easily-Customized Annotation System Powered by Efficiency
Enhancement Mechanisms. \sysname provides modular annotation units for building
customized annotation interfaces and also provides multiple back-end options
that suggest annotations using (1) multi-task active learning; (2) demographic
feature based active learning; (3) a prompt system that can query the API of
large language models. We conduct multiple experiments and user studies to
evaluate our system's flexibility and effectiveness. Our results show that our
system can meet the diverse needs of NLP researchers and significantly
accelerate the annotation process.