Identifying Spurious Correlations using Counterfactual Alignment
Joseph Paul Cohen Louis Blankemeier Akshay Chaudhari
Models driven by spurious correlations often yield poor generalization
performance. We propose the counterfactual alignment method to detect and
explore spurious correlations of black box classifiers. Counterfactual images
generated with respect to one classifier can be input into other classifiers to
see if they also induce changes in the outputs of these classifiers. The
relationship between these responses can be quantified and used to identify
specific instances where a spurious correlation exists as well as compute
aggregate statistics over a dataset. Our work demonstrates the ability to
detect spurious correlations in face attribute classifiers. This is validated
by observing intuitive trends in a face attribute classifier as well as
fabricating spurious correlations and detecting their presence, both visually
and quantitatively. Further, utilizing the CF alignment method, we demonstrate
that we can rectify spurious correlations identified in classifiers.