PG-LBO: Enhancing High-Dimensional Bayesian Optimization with
  Pseudo-Label and Gaussian Process Guidance
Taicai Chen Yue Duan Dong Li Lei Qi Yinghuan Shi Yang Gao
Variational Autoencoder based Bayesian Optimization (VAE-BO) has demonstrated
its excellent performance in addressing high-dimensional structured
optimization problems. However, current mainstream methods overlook the
potential of utilizing a pool of unlabeled data to construct the latent space,
while only concentrating on designing sophisticated models to leverage the
labeled data. Despite their effective usage of labeled data, these methods
often require extra network structures, additional procedure, resulting in
computational inefficiency. To address this issue, we propose a novel method to
effectively utilize unlabeled data with the guidance of labeled data.
Specifically, we tailor the pseudo-labeling technique from semi-supervised
learning to explicitly reveal the relative magnitudes of optimization objective
values hidden within the unlabeled data. Based on this technique, we assign
appropriate training weights to unlabeled data to enhance the construction of a
discriminative latent space. Furthermore, we treat the VAE encoder and the
Gaussian Process (GP) in Bayesian optimization as a unified deep kernel
learning process, allowing the direct utilization of labeled data, which we
term as Gaussian Process guidance. This directly and effectively integrates the
goal of improving GP accuracy into the VAE training, thereby guiding the
construction of the latent space. The extensive experiments demonstrate that
our proposed method outperforms existing VAE-BO algorithms in various
optimization scenarios. Our code will be published at
https://github.com/TaicaiChen/PG-LBO.