---
abstract: 'Estimating average causal effect (ACE) is useful whenever we want to know the effect of an intervention on a given outcome. In the absence of a randomized experiment, many methods such as stratification and inverse propensity weighting have been proposed to estimate ACE. However, it is hard to know which method is optimal for a given dataset or which hyperparameters to use for a chosen method. To this end, we provide a framework to characterize the loss of a causal inference method against the true ACE, by framing causal inference as a representation learning problem. We show that many popular methods, including back-door methods can be considered as weighting or representation learning algorithms, and provide general error bounds for their causal estimates. In addition, we consider the case when unobserved variables can confound the causal estimate and extend proposed bounds using principles of robust statistics, considering confounding as contamination under the Huber contamination model. These bounds are also estimable; as an example, we provide empirical bounds for the Inverse Propensity Weighting (IPW) estimator and show how the bounds can be used to optimize the threshold of clipping extreme propensity scores. Our work provides a new way to reason about competing estimators, and opens up the potential of deriving new methods by minimizing the proposed error bounds.'
author:
- |
    Rathin Desai\
    Microsoft Research India\
    Bangalore, Karnataka\
    `desairathin18@gmail.com` Amit Sharma\
    Microsoft Research India\
    Bangalore, Karnataka\
    `amshar@microsoft.com`
bibliography:
- 'ref.bib'
title: Quantifying Error in the Presence of Confounders for Causal Inference
---

Introduction
============

Consider the canonical causal inference problem where the goal is to find the effect of a treatment $T$ on some outcome $Y$, as shown in the structural causal model in Figure \[fig:source\]. This is relevant for estimating the effect of any fixed intervention, such as setting a system parameter, a medical intervention [@hernanbook] or a policy in social science settings [@morgan2015counterfactuals]. Here $W$ and $U$ are observed and unobserved common causes respectively, which affect the observed conditional distribution $\Pr(Y|T)$. To estimate the causal effect of $T$, methods typically condition on the observed common causes $W$ using the “back-door” formula [@pearl2009book], including methods such as stratification [@lunceford2004stratification], matching [@rubin1996matching], and inverse weighting [@rosenbaum1983central]. All of these methods work under the “ignorability" or the “selection on observables" assumption, where $U$ is assumed to have no effect once we condition on $W$ (i.e. $\Pr(Y|T,W) = \Pr(Y|T,W,U)$). In practice, however, ignorability is seldom satisfied and its violation can lead to significant errors, even changing the direction of the effect estimate. Because $U$ is unobserved, current methods provide no bounds on the error in a causal effect estimate when the assumption is violated. This makes it hard to compare methods for a given dataset, or to assess sensitivity of an estimate to unobserved confounding, except by simplistic simulations of the effect of $U$ [@rosenbaum2002observational].

In this paper, we provide a general framework for estimating error for causal inference methods, both in the presence and absence of $U$. Our insight is that the causal inference problem can be framed as a domain adaptation [@mansour2009domain] problem, where the target distribution is generated from a (hypothetical) randomized experiment on $T$, as shown in Figure \[fig:target1\]. Under this target distribution $P$, the observed effect $P(Y|T)$ is the same as the causal effect, $P(Y|do(T))$ since $T$ is no longer affected by $W$ or $U$ [@pearl2009book]. The goal of causal inference then is to use data from a source distribution $Q$ and estimate a function that approximates $P(Y|T)$. Alternatively, one can consider this as a task of learning an intermediate distribution R (or a *representation*), such that $R(Y|T)$ will be as close as possible to $P(Y|T)$. In this paper, using the lens of domain adaptation [@mansour2009domain], we provide bounds on the error of such estimators for the average causal effect (ACE), based on distance (bias) of the intermediate distribution $R$ from $P$ and variance in estimating it. In particular, we show that many causal inference methods such as stratification and inverse propensity weighting (IPW) can be considered as learning an intermediate representation.

When $U$ is ignorable, we provide bounds that separate out the effects of bias and variance in choosing $R$ and derive a procedure to estimate them from data. Empirical simulations show the value of the proposed error bound in evaluating different intermediate representations, and correspondingly, causal inference algorithms. For instance, our bound can be used to select the optimal threshold for clipping extreme probabilities—a common technique in weighting algorithms such as IPW—in order to minimize error. When $U$ is not ignorable, we utilize theory from robust estimators to characterize $U$’s effect on $Y$. The intuition is that confounding effect of $U$ on $Y$ can be considerd as contamination (noise) added to true function between $T$ and $Y$. In addition, we assume that this noise affects only a fraction of input data rows. Such an assumption is plausible whenever effect of $U$ is specific to certain units, for example, unobserved genes may only affect some people’s health outcome and be ignorable for other people. We use the Huber-contamination model [@huber1992robust] to model this noise, provide a robust estimator for the causal effect [@lai2016agnostic], and bound its error under the assumption that $U$ only affects a fraction of all outcomes $Y$. When such an assumption is not plausible, the bounds still allow us to study the sensitivity of the error as the amount of contamination (confounding) by $U$ is changed. Overall, our error bounds on causal estimators provides a principled way to compare different estimators and conduct sensitivity analysis of causal estimates with minimal parametric assumptions.

Conclusion
==========

We have provided general error bounds for any causal estimator that can be written as a weighted representation learner. The error naturally decomposes into the sampling error in estimating $R$ and measure of distance between the weighted distribution and target distribution $P$. The error terms also yield important insights for developing new methods by minimizing the error bounds.

Supplementary Materials
=======================
