Improving Event Definition Following For Zero-Shot Event Detection
Zefan Cai Po-Nien Kung Ashima Suvarna Mingyu Derek Ma Hritik Bansal Baobao Chang P. Jeffrey Brantingham Wei Wang Nanyun Peng
Existing approaches on zero-shot event detection usually train models on
datasets annotated with known event types, and prompt them with unseen event
definitions. These approaches yield sporadic successes, yet generally fall
short of expectations. In this work, we aim to improve zero-shot event
detection by training models to better follow event definitions. We hypothesize
that a diverse set of event types and definitions are the key for models to
learn to follow event definitions while existing event extraction datasets
focus on annotating many high-quality examples for a few event types. To verify
our hypothesis, we construct an automatically generated Diverse Event
Definition (DivED) dataset and conduct comparative studies. Our experiments
reveal that a large number of event types (200) and diverse event definitions
can significantly boost event extraction performance; on the other hand, the
performance does not scale with over ten examples per event type. Beyond
scaling, we incorporate event ontology information and hard-negative samples
during training, further boosting the performance. Based on these findings, we
fine-tuned a LLaMA-2-7B model on our DivED dataset, yielding performance that
surpasses SOTA large language models like GPT-3.5 across three open benchmarks
on zero-shot event detection.