Shortcuts for causal discovery of nonlinear models by score matching
Francesco Montagna Nicoletta Noceti Lorenzo Rosasco Francesco Locatello
The use of simulated data in the field of causal discovery is ubiquitous due
to the scarcity of annotated real data. Recently, Reisach et al., 2021
highlighted the emergence of patterns in simulated linear data, which displays
increasing marginal variance in the casual direction. As an ablation in their
experiments, Montagna et al., 2023 found that similar patterns may emerge in
nonlinear models for the variance of the score vector $\nabla \log
p_{\mathbf{X}}$, and introduced the ScoreSort algorithm. In this work, we
formally define and characterize this score-sortability pattern of nonlinear
additive noise models. We find that it defines a class of identifiable
(bivariate) causal models overlapping with nonlinear additive noise models. We
theoretically demonstrate the advantages of ScoreSort in terms of statistical
efficiency compared to prior state-of-the-art score matching-based methods and
empirically show the score-sortability of the most common synthetic benchmarks
in the literature. Our findings remark (1) the lack of diversity in the data as
an important limitation in the evaluation of nonlinear causal discovery
approaches, (2) the importance of thoroughly testing different settings within
a problem class, and (3) the importance of analyzing statistical properties in
causal discovery, where research is often limited to defining identifiability
conditions of the model.