Spurious Feature Diversification Improves Out-of-distribution
  Generalization
Yong Lin Lu Tan Yifan Hao Honam Wong Hanze Dong Weizhong Zhang Yujiu Yang Tong Zhang
Generalization to out-of-distribution (OOD) data is a critical challenge in
machine learning. Ensemble-based methods, like weight space ensembles that
interpolate model parameters, have been shown to achieve superior OOD
performance. However, the underlying mechanism for their effectiveness remains
unclear. In this study, we closely examine WiSE-FT, a popular weight space
ensemble method that interpolates between a pre-trained and a fine-tuned model.
We observe an unexpected phenomenon, in which WiSE-FT successfully corrects
many cases where each individual model makes incorrect predictions, which
contributes significantly to its OOD effectiveness. To gain further insights,
we conduct theoretical analysis in a multi-class setting with a large number of
spurious features. Our analysis predicts the above phenomenon and it further
shows that ensemble-based models reduce prediction errors in the OOD settings
by utilizing a more diverse set of spurious features. Contrary to the
conventional wisdom that focuses on learning invariant features for better OOD
performance, our findings suggest that incorporating a large number of diverse
spurious features weakens their individual contributions, leading to improved
overall OOD generalization performance. Empirically we demonstrate the
effectiveness of utilizing diverse spurious features on a MultiColorMNIST
dataset, and our experimental results are consistent with the theoretical
analysis. Building upon the new theoretical insights into the efficacy of
ensemble methods, we further identify an issue of WiSE-FT caused by the
overconfidence of fine-tuned models in OOD situations. This overconfidence
magnifies the fine-tuned model's incorrect prediction, leading to deteriorated
OOD ensemble performance. To remedy this problem, we propose a novel method
called BAlaNced averaGing (BANG), which significantly enhances the OOD
performance of WiSE-FT.