Rao-Blackwellising Bayesian Causal Inference
Christian Toth Christian Knoll Franz Pernkopf Robert Peharz
Bayesian causal inference, i.e., inferring a posterior over causal models for
the use in downstream causal reasoning tasks, poses a hard computational
inference problem that is little explored in literature. In this work, we
combine techniques from order-based MCMC structure learning with recent
advances in gradient-based graph learning into an effective Bayesian causal
inference framework. Specifically, we decompose the problem of inferring the
causal structure into (i) inferring a topological order over variables and (ii)
inferring the parent sets for each variable. When limiting the number of
parents per variable, we can exactly marginalise over the parent sets in
polynomial time. We further use Gaussian processes to model the unknown causal
mechanisms, which also allows their exact marginalisation. This introduces a
Rao-Blackwellization scheme, where all components are eliminated from the
model, except for the causal order, for which we learn a distribution via
gradient-based optimisation. The combination of Rao-Blackwellization with our
sequential inference procedure for causal orders yields state-of-the-art on
linear and non-linear additive noise benchmarks with scale-free and Erdos-Renyi
graph structures.