Robustly Improving Bandit Algorithms with Confounded and Selection
  Biased Offline Data: A Causal Approach
Wen Huang Xintao Wu
This paper studies bandit problems where an agent has access to offline data
that might be utilized to potentially improve the estimation of each arm's
reward distribution. A major obstacle in this setting is the existence of
compound biases from the observational data. Ignoring these biases and blindly
fitting a model with the biased data could even negatively affect the online
learning phase. In this work, we formulate this problem from a causal
perspective. First, we categorize the biases into confounding bias and
selection bias based on the causal structure they imply. Next, we extract the
causal bound for each arm that is robust towards compound biases from biased
observational data. The derived bounds contain the ground truth mean reward and
can effectively guide the bandit agent to learn a nearly-optimal decision
policy. We also conduct regret analysis in both contextual and non-contextual
bandit settings and show that prior causal bounds could help consistently
reduce the asymptotic regret.