Reinterpreting causal discovery as the task of predicting unobserved
  joint statistics
Dominik Janzing Philipp M. Faller Leena Chennuru Vankadara
If $X,Y,Z$ denote sets of random variables, two different data sources may
contain samples from $P_{X,Y}$ and $P_{Y,Z}$, respectively. We argue that
causal discovery can help inferring properties of the `unobserved joint
distributions' $P_{X,Y,Z}$ or $P_{X,Z}$. The properties may be conditional
independences (as in `integrative causal inference') or also quantitative
statements about dependences.
  More generally, we define a learning scenario where the input is a subset of
variables and the label is some statistical property of that subset. Sets of
jointly observed variables define the training points, while unobserved sets
are possible test points. To solve this learning task, we infer, as an
intermediate step, a causal model from the observations that then entails
properties of unobserved sets. Accordingly, we can define the VC dimension of a
class of causal models and derive generalization bounds for the predictions.
  Here, causal discovery becomes more modest and better accessible to empirical
tests than usual: rather than trying to find a causal hypothesis that is `true'
a causal hypothesis is {\it useful} whenever it correctly predicts statistical
properties of unobserved joint distributions. This way, a sparse causal graph
that omits weak influences may be more useful than a dense one (despite being
less accurate) because it is able to reconstruct the full joint distribution
from marginal distributions of smaller subsets.
  Within such a `pragmatic' application of causal discovery, some popular
heuristic approaches become justified in retrospect. It is, for instance,
allowed to infer DAGs from partial correlations instead of conditional
independences if the DAGs are only used to predict partial correlations.