PMBO: Enhancing Black-Box Optimization through Multivariate Polynomial
  Surrogates
Janina Schreiber Pau Batlle Damar Wicaksono Michael Hecht
We introduce a surrogate-based black-box optimization method, termed
Polynomial-model-based optimization (PMBO). The algorithm alternates polynomial
approximation with Bayesian optimization steps, using Gaussian processes to
model the error between the objective and its polynomial fit. We describe the
algorithmic design of PMBO and compare the results of the performance of PMBO
with several optimization methods for a set of analytic test functions.
  The results show that PMBO outperforms the classic Bayesian optimization and
is robust with respect to the choice of its correlation function family and its
hyper-parameter setting, which, on the contrary, need to be carefully tuned in
classic Bayesian optimization. Remarkably, PMBO performs comparably with
state-of-the-art evolutionary algorithms such as the Covariance Matrix
Adaptation -- Evolution Strategy (CMA-ES). This finding suggests that PMBO
emerges as the pivotal choice among surrogate-based optimization methods when
addressing low-dimensional optimization problems. Hereby, the simple nature of
polynomials opens the opportunity for interpretation and analysis of the
inferred surrogate model, providing a macroscopic perspective on the landscape
of the objective function.