Discovering environments with XRM
Mohammad Pezeshki Diane Bouchacourt Mark Ibrahim Nicolas Ballas Pascal Vincent David Lopez-Paz
Successful out-of-distribution generalization requires environment
annotations. Unfortunately, these are resource-intensive to obtain, and their
relevance to model performance is limited by the expectations and perceptual
biases of human annotators. Therefore, to enable robust AI systems across
applications, we must develop algorithms to automatically discover environments
inducing broad generalization. Current proposals, which divide examples based
on their training error, suffer from one fundamental problem. These methods add
hyper-parameters and early-stopping criteria that are impossible to tune
without a validation set with human-annotated environments, the very
information subject to discovery. In this paper, we propose
Cross-Risk-Minimization (XRM) to address this issue. XRM trains two twin
networks, each learning from one random half of the training data, while
imitating confident held-out mistakes made by its sibling. XRM provides a
recipe for hyper-parameter tuning, does not require early-stopping, and can
discover environments for all training and validation data. Domain
generalization algorithms built on top of XRM environments achieve oracle
worst-group-accuracy, solving a long-standing problem in out-of-distribution
generalization.