Dissecting vocabulary biases datasets through statistical testing and
  automated data augmentation for artifact mitigation in Natural Language
  Inference
Dat Thanh Nguyen
In recent years, the availability of large-scale annotated datasets, such as
the Stanford Natural Language Inference and the Multi-Genre Natural Language
Inference, coupled with the advent of pre-trained language models, has
significantly contributed to the development of the natural language inference
domain. However, these crowdsourced annotated datasets often contain biases or
dataset artifacts, leading to overestimated model performance and poor
generalization. In this work, we focus on investigating dataset artifacts and
developing strategies to address these issues. Through the utilization of a
novel statistical testing procedure, we discover a significant association
between vocabulary distribution and text entailment classes, emphasizing
vocabulary as a notable source of biases. To mitigate these issues, we propose
several automatic data augmentation strategies spanning character to word
levels. By fine-tuning the ELECTRA pre-trained language model, we compare the
performance of boosted models with augmented data against their baseline
counterparts. The experiments demonstrate that the proposed approaches
effectively enhance model accuracy and reduce biases by up to 0.66% and 1.14%,
respectively.