Embedded Named Entity Recognition using Probing Classifiers
Nicholas Popovič Michael Färber
Extracting semantic information from generated text is a useful tool for
applications such as automated fact checking or retrieval augmented generation.
Currently, this requires either separate models during inference, which
increases computational cost, or destructive fine-tuning of the language model.
Instead, we propose directly embedding information extraction capabilities into
pre-trained language models using probing classifiers, enabling efficient
simultaneous text generation and information extraction. For this, we introduce
an approach called EMBER and show that it enables named entity recognition in
decoder-only language models without fine-tuning them and while incurring
minimal additional computational cost at inference time. Specifically, our
experiments using GPT-2 show that EMBER maintains high token generation rates
during streaming text generation, with only a negligible decrease in speed of
around 1% compared to a 43.64% slowdown measured for a baseline using a
separate NER model. Code and data are available at
https://github.com/nicpopovic/EMBER.