OODRobustBench: benchmarking and analyzing adversarial robustness under
  distribution shift
Lin Li Yifei Wang Chawin Sitawarin Michael Spratling
Existing works have made great progress in improving adversarial robustness,
but typically test their method only on data from the same distribution as the
training data, i.e. in-distribution (ID) testing. As a result, it is unclear
how such robustness generalizes under input distribution shifts, i.e.
out-of-distribution (OOD) testing. This is a concerning omission as such
distribution shifts are unavoidable when methods are deployed in the wild. To
address this issue we propose a benchmark named OODRobustBench to
comprehensively assess OOD adversarial robustness using 23 dataset-wise shifts
(i.e. naturalistic shifts in input distribution) and 6 threat-wise shifts
(i.e., unforeseen adversarial threat models). OODRobustBench is used to assess
706 robust models using 60.7K adversarial evaluations. This large-scale
analysis shows that: 1) adversarial robustness suffers from a severe OOD
generalization issue; 2) ID robustness correlates strongly with OOD robustness,
in a positive linear way, under many distribution shifts. The latter enables
the prediction of OOD robustness from ID robustness. Based on this, we are able
to predict the upper limit of OOD robustness for existing robust training
schemes. The results suggest that achieving OOD robustness requires designing
novel methods beyond the conventional ones. Last, we discover that extra data,
data augmentation, advanced model architectures and particular regularization
approaches can improve OOD robustness. Noticeably, the discovered training
schemes, compared to the baseline, exhibit dramatically higher robustness under
threat shift while keeping high ID robustness, demonstrating new promising
solutions for robustness against both multi-attack and unforeseen attacks.