Logical Reasoning for Natural Language Inference Using Generated Facts
  as Atoms
Joe Stacey Pasquale Minervini Haim Dubossarsky Oana-Maria Camburu Marek Rei
State-of-the-art neural models can now reach human performance levels across
various natural language understanding tasks. However, despite this impressive
performance, models are known to learn from annotation artefacts at the expense
of the underlying task. While interpretability methods can identify influential
features for each prediction, there are no guarantees that these features are
responsible for the model decisions. Instead, we introduce a model-agnostic
logical framework to determine the specific information in an input responsible
for each model decision. This method creates interpretable Natural Language
Inference (NLI) models that maintain their predictive power. We achieve this by
generating facts that decompose complex NLI observations into individual
logical atoms. Our model makes predictions for each atom and uses logical rules
to decide the class of the observation based on the predictions for each atom.
We apply our method to the highly challenging ANLI dataset, where our framework
improves the performance of both a DeBERTa-base and BERT baseline. Our method
performs best on the most challenging examples, achieving a new
state-of-the-art for the ANLI round 3 test set. We outperform every baseline in
a reduced-data setting, and despite using no annotations for the generated
facts, our model predictions for individual facts align with human
expectations.