DANSK and DaCy 2.6.0: Domain Generalization of Danish Named Entity
  Recognition
Kenneth Enevoldsen Emil Trenckner Jessen Rebekah Baglini
Named entity recognition is one of the cornerstones of Danish NLP, essential
for language technology applications within both industry and research.
However, Danish NER is inhibited by a lack of available datasets. As a
consequence, no current models are capable of fine-grained named entity
recognition, nor have they been evaluated for potential generalizability issues
across datasets and domains. To alleviate these limitations, this paper
introduces: 1) DANSK: a named entity dataset providing for high-granularity
tagging as well as within-domain evaluation of models across a diverse set of
domains; 2) DaCy 2.6.0 that includes three generalizable models with
fine-grained annotation; and 3) an evaluation of current state-of-the-art
models' ability to generalize across domains. The evaluation of existing and
new models revealed notable performance discrepancies across domains, which
should be addressed within the field. Shortcomings of the annotation quality of
the dataset and its impact on model training and evaluation are also discussed.
Despite these limitations, we advocate for the use of the new dataset DANSK
alongside further work on the generalizability within Danish NER.