CollabKG: A Learnable Human-Machine-Cooperative Information Extraction
  Toolkit for (Event) Knowledge Graph Construction
Xiang Wei Yufeng Chen Ning Cheng Xingyu Cui Jinan Xu Wenjuan Han
In order to construct or extend entity-centric and event-centric knowledge
graphs (KG and EKG), the information extraction (IE) annotation toolkit is
essential. However, existing IE toolkits have several non-trivial problems,
such as not supporting multi-tasks, not supporting automatic updates. In this
work, we present CollabKG, a learnable human-machine-cooperative IE toolkit for
KG and EKG construction. Specifically, for the multi-task issue, CollabKG
unifies different IE subtasks, including named entity recognition (NER),
entity-relation triple extraction (RE), and event extraction (EE), and supports
both KG and EKG. Then, combining advanced prompting-based IE technology, the
human-machine-cooperation mechanism with LLMs as the assistant machine is
presented which can provide a lower cost as well as a higher performance.
Lastly, owing to the two-way interaction between the human and machine,
CollabKG with learning ability allows self-renewal. Besides, CollabKG has
several appealing features (e.g., customization, training-free, propagation,
etc.) that make the system powerful, easy-to-use, and high-productivity. We
holistically compare our toolkit with other existing tools on these features.
Human evaluation quantitatively illustrates that CollabKG significantly
improves annotation quality, efficiency, and stability simultaneously.