A robust assessment for invariant representations
Wenlu Tang Zicheng Liu
The performance of machine learning models can be impacted by changes in data
over time. A promising approach to address this challenge is invariant
learning, with a particular focus on a method known as invariant risk
minimization (IRM). This technique aims to identify a stable data
representation that remains effective with out-of-distribution (OOD) data.
While numerous studies have developed IRM-based methods adaptive to data
augmentation scenarios, there has been limited attention on directly assessing
how well these representations preserve their invariant performance under
varying conditions. In our paper, we propose a novel method to evaluate
invariant performance, specifically tailored for IRM-based methods. We
establish a bridge between the conditional expectation of an invariant
predictor across different environments through the likelihood ratio. Our
proposed criterion offers a robust basis for evaluating invariant performance.
We validate our approach with theoretical support and demonstrate its
effectiveness through extensive numerical studies.These experiments illustrate
how our method can assess the invariant performance of various representation
techniques.