Embedding Models for Supervised Automatic Extraction and Classification
  of Named Entities in Scientific Acknowledgements
Nina Smirnova Philipp Mayr
Acknowledgments in scientific papers may give an insight into aspects of the
scientific community, such as reward systems, collaboration patterns, and
hidden research trends. The aim of the paper is to evaluate the performance of
different embedding models for the task of automatic extraction and
classification of acknowledged entities from the acknowledgment text in
scientific papers. We trained and implemented a named entity recognition (NER)
task using the Flair NLP framework. The training was conducted using three
default Flair NER models with four differently-sized corpora and different
versions of the Flair NLP framework. The Flair Embeddings model trained on the
medium corpus with the latest FLAIR version showed the best accuracy of 0.79.
Expanding the size of a training corpus from very small to medium size
massively increased the accuracy of all training algorithms, but further
expansion of the training corpus did not bring further improvement. Moreover,
the performance of the model slightly deteriorated. Our model is able to
recognize six entity types: funding agency, grant number, individuals,
university, corporation, and miscellaneous. The model works more precisely for
some entity types than for others; thus, individuals and grant numbers showed a
very good F1-Score over 0.9. Most of the previous works on acknowledgment
analysis were limited by the manual evaluation of data and therefore by the
amount of processed data. This model can be applied for the comprehensive
analysis of acknowledgment texts and may potentially make a great contribution
to the field of automated acknowledgment analysis.