LegalPro-BERT: Classification of Legal Provisions by fine-tuning BERT
  Large Language Model
Amit Tewari
A contract is a type of legal document commonly used in organizations.
Contract review is an integral and repetitive process to avoid business risk
and liability. Contract analysis requires the identification and classification
of key provisions and paragraphs within an agreement. Identification and
validation of contract clauses can be a time-consuming and challenging task
demanding the services of trained and expensive lawyers, paralegals or other
legal assistants. Classification of legal provisions in contracts using
artificial intelligence and natural language processing is complex due to the
requirement of domain-specialized legal language for model training and the
scarcity of sufficient labeled data in the legal domain. Using general-purpose
models is not effective in this context due to the use of specialized legal
vocabulary in contracts which may not be recognized by a general model. To
address this problem, we propose the use of a pre-trained large language model
which is subsequently calibrated on legal taxonomy. We propose LegalPro-BERT, a
BERT transformer architecture model that we fine-tune to efficiently handle
classification task for legal provisions. We conducted experiments to measure
and compare metrics with current benchmark results. We found that LegalPro-BERT
outperforms the previous benchmark used for comparison in this research.