Generating Clarification Questions for Disambiguating Contracts
Anmol Singhal Chirag Jain Preethu Rose Anish Arkajyoti Chakraborty Smita Ghaisas
Enterprises frequently enter into commercial contracts that can serve as
vital sources of project-specific requirements. Contractual clauses are
obligatory, and the requirements derived from contracts can detail the
downstream implementation activities that non-legal stakeholders, including
requirement analysts, engineers, and delivery personnel, need to conduct.
However, comprehending contracts is cognitively demanding and error-prone for
such stakeholders due to the extensive use of Legalese and the inherent
complexity of contract language. Furthermore, contracts often contain
ambiguously worded clauses to ensure comprehensive coverage. In contrast,
non-legal stakeholders require a detailed and unambiguous comprehension of
contractual clauses to craft actionable requirements. In this work, we
introduce a novel legal NLP task that involves generating clarification
questions for contracts. These questions aim to identify contract ambiguities
on a document level, thereby assisting non-legal stakeholders in obtaining the
necessary details for eliciting requirements. This task is challenged by three
core issues: (1) data availability, (2) the length and unstructured nature of
contracts, and (3) the complexity of legal text. To address these issues, we
propose ConRAP, a retrieval-augmented prompting framework for generating
clarification questions to disambiguate contractual text. Experiments conducted
on contracts sourced from the publicly available CUAD dataset show that ConRAP
with ChatGPT can detect ambiguities with an F2 score of 0.87. 70% of the
generated clarification questions are deemed useful by human evaluators.