Boosted Control Functions
Nicola Gnecco Jonas Peters Sebastian Engelke Niklas Pfister
Modern machine learning methods and the availability of large-scale data
opened the door to accurately predict target quantities from large sets of
covariates. However, existing prediction methods can perform poorly when the
training and testing data are different, especially in the presence of hidden
confounding. While hidden confounding is well studied for causal effect
estimation (e.g., instrumental variables), this is not the case for prediction
tasks. This work aims to bridge this gap by addressing predictions under
different training and testing distributions in the presence of unobserved
confounding. In particular, we establish a novel connection between the field
of distribution generalization from machine learning, and simultaneous equation
models and control function from econometrics. Central to our contribution are
simultaneous equation models for distribution generalization (SIMDGs) which
describe the data-generating process under a set of distributional shifts.
Within this framework, we propose a strong notion of invariance for a
predictive model and compare it with existing (weaker) versions. Building on
the control function approach from instrumental variable regression, we propose
the boosted control function (BCF) as a target of inference and prove its
ability to successfully predict even in intervened versions of the underlying
SIMDG. We provide necessary and sufficient conditions for identifying the BCF
and show that it is worst-case optimal. We introduce the ControlTwicing
algorithm to estimate the BCF and analyze its predictive performance on
simulated and real world data.