DALE: Generative Data Augmentation for Low-Resource Legal NLP
Sreyan Ghosh Chandra Kiran Evuru Sonal Kumar S Ramaneswaran S Sakshi Utkarsh Tyagi Dinesh Manocha
We present DALE, a novel and effective generative Data Augmentation framework
for low-resource LEgal NLP. DALE addresses the challenges existing frameworks
pose in generating effective data augmentations of legal documents - legal
language, with its specialized vocabulary and complex semantics, morphology,
and syntax, does not benefit from data augmentations that merely rephrase the
source sentence. To address this, DALE, built on an Encoder-Decoder Language
Model, is pre-trained on a novel unsupervised text denoising objective based on
selective masking - our masking strategy exploits the domain-specific language
characteristics of templatized legal documents to mask collocated spans of
text. Denoising these spans helps DALE acquire knowledge about legal concepts,
principles, and language usage. Consequently, it develops the ability to
generate coherent and diverse augmentations with novel contexts. Finally, DALE
performs conditional generation to generate synthetic augmentations for
low-resource Legal NLP tasks. We demonstrate the effectiveness of DALE on 13
datasets spanning 6 tasks and 4 low-resource settings. DALE outperforms all our
baselines, including LLMs, qualitatively and quantitatively, with improvements
of 1%-50%.