PAVI: Plate-Amortized Variational Inference
Louis Rouillard Alexandre Le Bris Thomas Moreau Demian Wassermann
Given observed data and a probabilistic generative model, Bayesian inference
searches for the distribution of the model's parameters that could have yielded
the data. Inference is challenging for large population studies where millions
of measurements are performed over a cohort of hundreds of subjects, resulting
in a massive parameter space. This large cardinality renders off-the-shelf
Variational Inference (VI) computationally impractical.
  In this work, we design structured VI families that efficiently tackle large
population studies. Our main idea is to share the parameterization and learning
across the different i.i.d. variables in a generative model, symbolized by the
model's \textit{plates}. We name this concept \textit{plate amortization}.
Contrary to off-the-shelf stochastic VI, which slows down inference, plate
amortization results in orders of magnitude faster to train variational
distributions.
  Applied to large-scale hierarchical problems, PAVI yields expressive,
parsimoniously parameterized VI with an affordable training time. This faster
convergence effectively unlocks inference in those large regimes. We illustrate
the practical utility of PAVI through a challenging Neuroimaging example
featuring 400 million latent parameters, demonstrating a significant step
towards scalable and expressive Variational Inference.