Amortised Inference in Neural Networks for Small-Scale Probabilistic
  Meta-Learning
Matthew Ashman Tommy Rochussen Adrian Weller
The global inducing point variational approximation for BNNs is based on
using a set of inducing inputs to construct a series of conditional
distributions that accurately approximate the conditionals of the true
posterior distribution. Our key insight is that these inducing inputs can be
replaced by the actual data, such that the variational distribution consists of
a set of approximate likelihoods for each datapoint. This structure lends
itself to amortised inference, in which the parameters of each approximate
likelihood are obtained by passing each datapoint through a meta-model known as
the inference network. By training this inference network across related
datasets, we can meta-learn Bayesian inference over task-specific BNNs.