Can predictive models be used for causal inference?
Maximilian Pichler Florian Hartig
Supervised machine learning (ML) and deep learning (DL) algorithms excel at
predictive tasks, but it is commonly assumed that they often do so by
exploiting non-causal correlations, which may limit both interpretability and
generalizability. Here, we show that this trade-off between explanation and
prediction is not as deep and fundamental as expected. Whereas ML and DL
algorithms will indeed tend to use non-causal features for prediction when fed
indiscriminately with all data, it is possible to constrain the learning
process of any ML and DL algorithm by selecting features according to Pearl's
backdoor adjustment criterion. In such a situation, some algorithms, in
particular deep neural networks, can provide near unbiased effect estimates
under feature collinearity. Remaining biases are explained by the specific
algorithmic structures as well as hyperparameter choice. Consequently, optimal
hyperparameter settings are different when tuned for prediction or inference,
confirming the general expectation of a trade-off between prediction and
explanation. However, the effect of this trade-off is small compared to the
effect of a causally constrained feature selection. Thus, once the causal
relationship between the features is accounted for, the difference between
prediction and explanation may be much smaller than commonly assumed. We also
show that such causally constrained models generalize better to new data with
altered collinearity structures, suggesting generalization failure may often be
due to a lack of causal learning. Our results not only provide a perspective
for using ML for inference of (causal) effects but also help to improve the
generalizability of fitted ML and DL models to new data.