Improving Access to Justice for the Indian Population: A Benchmark for
  Evaluating Translation of Legal Text to Indian Languages
Sayan Mahapatra Debtanu Datta Shubham Soni Adrijit Goswami Saptarshi Ghosh
Most legal text in the Indian judiciary is written in complex English due to
historical reasons. However, only about 10% of the Indian population is
comfortable in reading English. Hence legal text needs to be made available in
various Indian languages, possibly by translating the available legal text from
English. Though there has been a lot of research on translation to and between
Indian languages, to our knowledge, there has not been much prior work on such
translation in the legal domain. In this work, we construct the first
high-quality legal parallel corpus containing aligned text units in English and
nine Indian languages, that includes several low-resource languages. We also
benchmark the performance of a wide variety of Machine Translation (MT) systems
over this corpus, including commercial MT systems, open-source MT systems and
Large Language Models. Through a comprehensive survey by Law practitioners, we
check how satisfied they are with the translations by some of these MT systems,
and how well automatic MT evaluation metrics agree with the opinions of Law
practitioners.