Causality-oriented robustness: exploiting general additive interventions
Xinwei Shen Peter Bühlmann Armeen Taeb
Since distribution shifts are common in real-world applications, there is a
pressing need for developing prediction models that are robust against such
shifts. Existing frameworks, such as empirical risk minimization or
distributionally robust optimization, either lack generalizability for unseen
distributions or rely on postulated distance measures. Alternatively, causality
offers a data-driven and structural perspective to robust predictions. However,
the assumptions necessary for causal inference can be overly stringent, and the
robustness offered by such causal models often lacks flexibility. In this
paper, we focus on causality-oriented robustness and propose Distributional
Robustness via Invariant Gradients (DRIG), a method that exploits general
additive interventions in training data for robust predictions against unseen
interventions, and naturally interpolates between in-distribution prediction
and causality. In a linear setting, we prove that DRIG yields predictions that
are robust among a data-dependent class of distribution shifts. Furthermore, we
show that our framework includes anchor regression (Rothenh\"ausler et al.\
2021) as a special case, and that it yields prediction models that protect
against more diverse perturbations. We extend our approach to the
semi-supervised domain adaptation setting to further improve prediction
performance. Finally, we empirically validate our methods on synthetic
simulations and on single-cell data.