Anonymity at Risk? Assessing Re-Identification Capabilities of Large
  Language Models
Alex Nyffenegger Matthias Stürmer Joel Niklaus
Anonymity of both natural and legal persons in court rulings is a critical
aspect of privacy protection in the European Union and Switzerland. With the
advent of LLMs, concerns about large-scale re-identification of anonymized
persons are growing. In accordance with the Federal Supreme Court of
Switzerland, we explore the potential of LLMs to re-identify individuals in
court rulings by constructing a proof-of-concept using actual legal data from
the Swiss federal supreme court. Following the initial experiment, we
constructed an anonymized Wikipedia dataset as a more rigorous testing ground
to further investigate the findings. With the introduction and application of
the new task of re-identifying people in texts, we also introduce new metrics
to measure performance. We systematically analyze the factors that influence
successful re-identifications, identifying model size, input length, and
instruction tuning among the most critical determinants. Despite high
re-identification rates on Wikipedia, even the best LLMs struggled with court
decisions. The complexity is attributed to the lack of test datasets, the
necessity for substantial training resources, and data sparsity in the
information used for re-identification. In conclusion, this study demonstrates
that re-identification using LLMs may not be feasible for now, but as the
proof-of-concept on Wikipedia showed, it might become possible in the future.
We hope that our system can help enhance the confidence in the security of
anonymized decisions, thus leading to the courts being more confident to
publish decisions.