NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data
Sergei Bogdanov Alexandre Constantin Timothée Bernard Benoit Crabbé Etienne Bernard
Large Language Models (LLMs) have shown impressive abilities in data
annotation, opening the way for new approaches to solve classic NLP problems.
In this paper, we show how to use LLMs to create NuNER, a compact language
representation model specialized in the Named Entity Recognition (NER) task.
NuNER can be fine-tuned to solve downstream NER problems in a data-efficient
way, outperforming similar-sized foundation models in the few-shot regime and
competing with much larger LLMs. We find that the size and entity-type
diversity of the pre-training dataset are key to achieving good performance. We
view NuNER as a member of the broader family of task-specific foundation
models, recently unlocked by LLMs.