Automated Refugee Case Analysis: An NLP Pipeline for Supporting Legal
  Practitioners
Claire Barale Michael Rovatsos Nehal Bhuta
In this paper, we introduce an end-to-end pipeline for retrieving,
processing, and extracting targeted information from legal cases. We
investigate an under-studied legal domain with a case study on refugee law in
Canada. Searching case law for past similar cases is a key part of legal work
for both lawyers and judges, the potential end-users of our prototype. While
traditional named-entity recognition labels such as dates provide meaningful
information in legal work, we propose to extend existing models and retrieve a
total of 19 useful categories of items from refugee cases. After creating a
novel data set of cases, we perform information extraction based on
state-of-the-art neural named-entity recognition (NER). We test different
architectures including two transformer models, using contextual and
non-contextual embeddings, and compare general purpose versus domain-specific
pre-training. The results demonstrate that models pre-trained on legal data
perform best despite their smaller size, suggesting that domain matching had a
larger effect than network architecture. We achieve a F1 score above 90% on
five of the targeted categories and over 80% on four further categories.