Implicit Maximum a Posteriori Filtering via Adaptive Optimization
Gianluca M. Bencomo Jake C. Snell Thomas L. Griffiths
Bayesian filtering approximates the true underlying behavior of a
time-varying system by inverting an explicit generative model to convert noisy
measurements into state estimates. This process typically requires either
storage, inversion, and multiplication of large matrices or Monte Carlo
estimation, neither of which are practical in high-dimensional state spaces
such as the weight spaces of artificial neural networks. Here, we frame the
standard Bayesian filtering problem as optimization over a time-varying
objective. Instead of maintaining matrices for the filtering equations or
simulating particles, we specify an optimizer that defines the Bayesian filter
implicitly. In the linear-Gaussian setting, we show that every Kalman filter
has an equivalent formulation using K steps of gradient descent. In the
nonlinear setting, our experiments demonstrate that our framework results in
filters that are effective, robust, and scalable to high-dimensional systems,
comparing well against the standard toolbox of Bayesian filtering solutions. We
suggest that it is easier to fine-tune an optimizer than it is to specify the
correct filtering equations, making our framework an attractive option for
high-dimensional filtering problems.