Towards Consistent Language Models Using Declarative Constraints
Jasmin Mousavi Arash Termehchy
Large language models have shown unprecedented abilities in generating
linguistically coherent and syntactically correct natural language output.
However, they often return incorrect and inconsistent answers to input
questions. Due to the complexity and uninterpretability of the internally
learned representations, it is challenging to modify language models such that
they provide correct and consistent results. The data management community has
developed various methods and tools for providing consistent answers over
inconsistent datasets. In these methods, users specify the desired properties
of data in a domain in the form of high-level declarative constraints. This
approach has provided usable and scalable methods to delivering consistent
information from inconsistent datasets. We aim to build upon this success and
leverage these methods to modify language models such that they deliver
consistent and accurate results. We investigate the challenges of using these
ideas to obtain consistent and relevant answers from language models and report
some preliminary empirical studies.