Convergence of flow-based generative models via proximal gradient
  descent in Wasserstein space
Xiuyuan Cheng Jianfeng Lu Yixin Tan Yao Xie
Flow-based generative models enjoy certain advantages in computing the data
generation and the likelihood, and have recently shown competitive empirical
performance. Compared to the accumulating theoretical studies on related
score-based diffusion models, analysis of flow-based models, which are
deterministic in both forward (data-to-noise) and reverse (noise-to-data)
directions, remain sparse. In this paper, we provide a theoretical guarantee of
generating data distribution by a progressive flow model, the so-called JKO
flow model, which implements the Jordan-Kinderleherer-Otto (JKO) scheme in a
normalizing flow network. Leveraging the exponential convergence of the
proximal gradient descent (GD) in Wasserstein space, we prove the
Kullback-Leibler (KL) guarantee of data generation by a JKO flow model to be
$O(\varepsilon^2)$ when using $N \lesssim \log (1/\varepsilon)$ many JKO steps
($N$ Residual Blocks in the flow) where $\varepsilon $ is the error in the
per-step first-order condition. The assumption on data density is merely a
finite second moment, and the theory extends to data distributions without
density and when there are inversion errors in the reverse process where we
obtain KL-$W_2$ mixed error guarantees. The non-asymptotic convergence rate of
the JKO-type $W_2$-proximal GD is proved for a general class of convex
objective functionals that includes the KL divergence as a special case, which
can be of independent interest.