High-dimensional Bayesian Optimization via Covariance Matrix Adaptation
  Strategy
Lam Ngo Huong Ha Jeffrey Chan Vu Nguyen Hongyu Zhang
Bayesian Optimization (BO) is an effective method for finding the global
optimum of expensive black-box functions. However, it is well known that
applying BO to high-dimensional optimization problems is challenging. To
address this issue, a promising solution is to use a local search strategy that
partitions the search domain into local regions with high likelihood of
containing the global optimum, and then use BO to optimize the objective
function within these regions. In this paper, we propose a novel technique for
defining the local regions using the Covariance Matrix Adaptation (CMA)
strategy. Specifically, we use CMA to learn a search distribution that can
estimate the probabilities of data points being the global optimum of the
objective function. Based on this search distribution, we then define the local
regions consisting of data points with high probabilities of being the global
optimum. Our approach serves as a meta-algorithm as it can incorporate existing
black-box BO optimizers, such as BO, TuRBO, and BAxUS, to find the global
optimum of the objective function within our derived local regions. We evaluate
our proposed method on various benchmark synthetic and real-world problems. The
results demonstrate that our method outperforms existing state-of-the-art
techniques.