Variational Sequential Optimal Experimental Design using Reinforcement
  Learning
Wanggang Shen Jiayuan Dong Xun Huan
We introduce variational sequential Optimal Experimental Design (vsOED), a
new method for optimally designing a finite sequence of experiments under a
Bayesian framework and with information-gain utilities. Specifically, we adopt
a lower bound estimator for the expected utility through variational
approximation to the Bayesian posteriors. The optimal design policy is solved
numerically by simultaneously maximizing the variational lower bound and
performing policy gradient updates. We demonstrate this general methodology for
a range of OED problems targeting parameter inference, model discrimination,
and goal-oriented prediction. These cases encompass explicit and implicit
likelihoods, nuisance parameters, and physics-based partial differential
equation models. Our vsOED results indicate substantially improved sample
efficiency and reduced number of forward model simulations compared to previous
sequential design algorithms.