Sequential transport maps using SoS density estimation and
  $α$-divergences
Benjamin Zanger Tiangang Cui Martin Schreiber Olivier Zahm
Transport-based density estimation methods are receiving growing interest
because of their ability to efficiently generate samples from the approximated
density. We further invertigate the sequential transport maps framework
proposed from arXiv:2106.04170 arXiv:2303.02554, which builds on a sequence of
composed Knothe-Rosenblatt (KR) maps. Each of those maps are built by first
estimating an intermediate density of moderate complexity, and then by
computing the exact KR map from a reference density to the precomputed
approximate density. In our work, we explore the use of Sum-of-Squares (SoS)
densities and $\alpha$-divergences for approximating the intermediate
densities. Combining SoS densities with $\alpha$-divergence interestingly
yields convex optimization problems which can be efficiently solved using
semidefinite programming. The main advantage of $\alpha$-divergences is to
enable working with unnormalized densities, which provides benefits both
numerically and theoretically. In particular, we provide two new convergence
analyses of the sequential transport maps: one based on a triangle-like
inequality and the second on information geometric properties of
$\alpha$-divergences for unnormalizied densities. The choice of intermediate
densities is also crucial for the efficiency of the method. While tempered (or
annealed) densities are the state-of-the-art, we introduce diffusion-based
intermediate densities which permits to approximate densities known from
samples only. Such intermediate densities are well-established in machine
learning for generative modeling. Finally we propose and try different
low-dimensional maps (or lazy maps) for dealing with high-dimensional problems
and numerically demonstrate our methods on several benchmarks, including
Bayesian inference problems and unsupervised learning task.