Mastering the exploration-exploitation trade-off in Bayesian
  Optimization
Antonio Candelieri
Gaussian Process based Bayesian Optimization is a well-known sample efficient
sequential strategy for globally optimizing black-box, expensive, and
multi-extremal functions. The role of the Gaussian Process is to provide a
probabilistic approximation of the unknown function, depending on the
sequentially collected observations, while an acquisition function drives the
choice of the next solution to evaluate, balancing between exploration and
exploitation, depending on the current Gaussian Process model. Despite the huge
effort of the scientific community in defining effective
exploration-exploitation mechanisms, we are still far away from the master
acquisition function. This paper merges the most relevant results and insights
from both algorithmic and human search strategies to propose a novel
acquisition function, mastering the trade-off between explorative and
exploitative choices, adaptively. We compare the proposed acquisition function
on a number of test functions and against different state-of-the-art ones,
which are instead based on prefixed or random scheduling between exploration
and exploitation. A Pareto analysis is performed with respect to two
(antagonistic) goals: convergence to the optimum and exploration capability.
Results empirically prove that the proposed acquisition function is almost
always Pareto optimal and also the most balanced trade-off between the two
goals.