Unveiling the Potential of Robustness in Evaluating Causal Inference
  Models
Yiyan Huang Cheuk Hang Leung Siyi Wang Yijun Li Qi Wu
The growing demand for personalized decision-making has led to a surge of
interest in estimating the Conditional Average Treatment Effect (CATE). The
intersection of machine learning and causal inference has yielded various
effective CATE estimators. However, deploying these estimators in practice is
often hindered by the absence of counterfactual labels, making it challenging
to select the desirable CATE estimator using conventional model selection
procedures like cross-validation. Existing approaches for CATE estimator
selection, such as plug-in and pseudo-outcome metrics, face two inherent
challenges. Firstly, they are required to determine the metric form and the
underlying machine learning models for fitting nuisance parameters or plug-in
learners. Secondly, they lack a specific focus on selecting a robust estimator.
To address these challenges, this paper introduces a novel approach, the
Distributionally Robust Metric (DRM), for CATE estimator selection. The
proposed DRM not only eliminates the need to fit additional models but also
excels at selecting a robust CATE estimator. Experimental studies demonstrate
the efficacy of the DRM method, showcasing its consistent effectiveness in
identifying superior estimators while mitigating the risk of selecting inferior
ones.