Keeping Up with the Language Models: Robustness-Bias Interplay in NLI
  Data and Models
Ioana Baldini Chhavi Yadav Payel Das Kush R. Varshney
Auditing unwanted social bias in language models (LMs) is inherently hard due
to the multidisciplinary nature of the work. In addition, the rapid evolution
of LMs can make benchmarks irrelevant in no time. Bias auditing is further
complicated by LM brittleness: when a presumably biased outcome is observed, is
it due to model bias or model brittleness? We propose enlisting the models
themselves to help construct bias auditing datasets that remain challenging,
and introduce bias measures that distinguish between types of model errors.
First, we extend an existing bias benchmark for NLI (BBNLI) using a combination
of LM-generated lexical variations, adversarial filtering, and human
validation. We demonstrate that the newly created dataset (BBNLInext) is more
challenging than BBNLI: on average, BBNLI-next reduces the accuracy of
state-of-the-art NLI models from 95.3%, as observed by BBNLI, to 58.6%. Second,
we employ BBNLI-next to showcase the interplay between robustness and bias, and
the subtlety in differentiating between the two. Third, we point out
shortcomings in current bias scores used in the literature and propose bias
measures that take into account pro-/anti-stereotype bias and model
brittleness. We will publicly release the BBNLI-next dataset to inspire
research on rapidly expanding benchmarks to keep up with model evolution, along
with research on the robustness-bias interplay in bias auditing.
  Note: This paper contains offensive text examples.