Parallelized Acquisition for Active Learning using Monte Carlo Sampling
Jesús Torrado Nils Schöneberg Jonas El Gammal
Bayesian inference remains one of the most important tool-kits for any
scientist, but increasingly expensive likelihood functions are required for
ever-more complex experiments, raising the cost of generating a Monte Carlo
sample of the posterior. Recent attention has been directed towards the use of
emulators of the posterior based on Gaussian Process (GP) regression combined
with active sampling to achieve comparable precision with far fewer costly
likelihood evaluations. Key to this approach is the batched acquisition of
proposals, so that the true posterior can be evaluated in parallel. This is
usually achieved via sequential maximization of the highly multimodal
acquisition function. Unfortunately, this approach parallelizes poorly and is
prone to getting stuck in local maxima. Our approach addresses this issue by
generating nearly-optimal batches of candidates using an almost-embarrassingly
parallel Nested Sampler on the mean prediction of the GP. The resulting
nearly-sorted Monte Carlo sample is used to generate a batch of candidates
ranked according to their sequentially conditioned acquisition function values
at little cost. The final sample can also be used for inferring marginal
quantities. Our proposed implementation (NORA) demonstrates comparable accuracy
to sequential conditioned acquisition optimization and efficient
parallelization in various synthetic and cosmological inference problems.