Adaptive Optimization for Prediction with Missing Data
Dimitris Bertsimas Arthur Delarue Jean Pauphilet
When training predictive models on data with missing entries, the most widely
used and versatile approach is a pipeline technique where we first impute
missing entries and then compute predictions. In this paper, we view prediction
with missing data as a two-stage adaptive optimization problem and propose a
new class of models, adaptive linear regression models, where the regression
coefficients adapt to the set of observed features. We show that some adaptive
linear regression models are equivalent to learning an imputation rule and a
downstream linear regression model simultaneously instead of sequentially. We
leverage this joint-impute-then-regress interpretation to generalize our
framework to non-linear models. In settings where data is strongly not missing
at random, our methods achieve a 2-10% improvement in out-of-sample accuracy.