ConstraintChecker: A Plugin for Large Language Models to Reason on
  Commonsense Knowledge Bases
Quyet V. Do Tianqing Fang Shizhe Diao Zhaowei Wang Yangqiu Song
Reasoning over Commonsense Knowledge Bases (CSKB), i.e. CSKB reasoning, has
been explored as a way to acquire new commonsense knowledge based on reference
knowledge in the original CSKBs and external prior knowledge. Despite the
advancement of Large Language Models (LLM) and prompt engineering techniques in
various reasoning tasks, they still struggle to deal with CSKB reasoning. One
of the problems is that it is hard for them to acquire explicit relational
constraints in CSKBs from only in-context exemplars, due to a lack of symbolic
reasoning capabilities (Bengio et al., 2021). To this end, we proposed
**ConstraintChecker**, a plugin over prompting techniques to provide and check
explicit constraints. When considering a new knowledge instance,
ConstraintChecker employs a rule-based module to produce a list of constraints,
then it uses a zero-shot learning module to check whether this knowledge
instance satisfies all constraints. The acquired constraint-checking result is
then aggregated with the output of the main prompting technique to produce the
final output. Experimental results on CSKB Reasoning benchmarks demonstrate the
effectiveness of our method by bringing consistent improvements over all
prompting methods. Codes and data are available at
\url{https://github.com/HKUST-KnowComp/ConstraintChecker}.