Poisson Process for Bayesian Optimization
Xiaoxing Wang Jiaxing Li Chao Xue Wei Liu Weifeng Liu Xiaokang Yang Junchi Yan Dacheng Tao
BayesianOptimization(BO) is a sample-efficient black-box optimizer, and
extensive methods have been proposed to build the absolute function response of
the black-box function through a probabilistic surrogate model, including
Tree-structured Parzen Estimator (TPE), random forest (SMAC), and Gaussian
process (GP). However, few methods have been explored to estimate the relative
rankings of candidates, which can be more robust to noise and have better
practicality than absolute function responses, especially when the function
responses are intractable but preferences can be acquired. To this end, we
propose a novel ranking-based surrogate model based on the Poisson process and
introduce an efficient BO framework, namely Poisson Process Bayesian
Optimization (PoPBO). Two tailored acquisition functions are further derived
from classic LCB and EI to accommodate it. Compared to the classic GP-BO
method, our PoPBO has lower computation costs and better robustness to noise,
which is verified by abundant experiments. The results on both simulated and
real-world benchmarks, including hyperparameter optimization (HPO) and neural
architecture search (NAS), show the effectiveness of PoPBO.