Questioning Biases in Case Judgment Summaries: Legal Datasets or Large
  Language Models?
Aniket Deroy Subhankar Maity
The evolution of legal datasets and the advent of large language models
(LLMs) have significantly transformed the legal field, particularly in the
generation of case judgment summaries. However, a critical concern arises
regarding the potential biases embedded within these summaries. This study
scrutinizes the biases present in case judgment summaries produced by legal
datasets and large language models. The research aims to analyze the impact of
biases on legal decision making. By interrogating the accuracy, fairness, and
implications of biases in these summaries, this study contributes to a better
understanding of the role of technology in legal contexts and the implications
for justice systems worldwide. In this study, we investigate biases wrt
Gender-related keywords, Race-related keywords, Keywords related to crime
against women, Country names and religious keywords. The study shows
interesting evidences of biases in the outputs generated by the large language
models and pre-trained abstractive summarization models. The reasoning behind
these biases needs further studies.