Calibrated Propensity Scores for Causal Effect Estimation
Shachi Deshpande Volodymyr Kuleshov
Propensity scores are commonly used to balance observed covariates while
estimating treatment effects. Estimates obtained through propensity score
weighing can be biased when the propensity score model cannot learn the true
treatment assignment mechanism. We argue that the probabilistic output of a
learned propensity score model should be calibrated, i.e. a predictive
treatment probability of 90% should correspond to 90% of individuals being
assigned the treatment group. We propose simple recalibration techniques to
ensure this property. We investigate the theoretical properties of a calibrated
propensity score model and its role in unbiased treatment effect estimation. We
demonstrate improved causal effect estimation with calibrated propensity scores
in several tasks including high-dimensional genome-wide association studies,
where we also show reduced computational requirements when calibration is
applied to simpler propensity score models.