Computing Marginal and Conditional Divergences between Decomposable
  Models with Applications
Loong Kuan Lee Geoffrey I. Webb Daniel F. Schmidt Nico Piatkowski
The ability to compute the exact divergence between two high-dimensional
distributions is useful in many applications but doing so naively is
intractable. Computing the alpha-beta divergence -- a family of divergences
that includes the Kullback-Leibler divergence and Hellinger distance -- between
the joint distribution of two decomposable models, i.e chordal Markov networks,
can be done in time exponential in the treewidth of these models. However,
reducing the dissimilarity between two high-dimensional objects to a single
scalar value can be uninformative. Furthermore, in applications such as
supervised learning, the divergence over a conditional distribution might be of
more interest. Therefore, we propose an approach to compute the exact
alpha-beta divergence between any marginal or conditional distribution of two
decomposable models. Doing so tractably is non-trivial as we need to decompose
the divergence between these distributions and therefore, require a
decomposition over the marginal and conditional distributions of these models.
Consequently, we provide such a decomposition and also extend existing work to
compute the marginal and conditional alpha-beta divergence between these
decompositions. We then show how our method can be used to analyze
distributional changes by first applying it to a benchmark image dataset.
Finally, based on our framework, we propose a novel way to quantify the error
in contemporary superconducting quantum computers. Code for all experiments is
available at: https://lklee.dev/pub/2023-icdm/code