Decreasing the Computing Time of Bayesian Optimization using
  Generalizable Memory Pruning
Alexander E. Siemenn Tonio Buonassisi
Bayesian optimization (BO) suffers from long computing times when processing
highly-dimensional or large data sets. These long computing times are a result
of the Gaussian process surrogate model having a polynomial time complexity
with the number of experiments. Running BO on high-dimensional or massive data
sets becomes intractable due to this time complexity scaling, in turn,
hindering experimentation. Alternative surrogate models have been developed to
reduce the computing utilization of the BO procedure, however, these methods
require mathematical alteration of the inherit surrogate function, pigeonholing
use into only that function. In this paper, we demonstrate a generalizable BO
wrapper of memory pruning and bounded optimization, capable of being used with
any surrogate model and acquisition function. Using this memory pruning
approach, we show a decrease in wall-clock computing times per experiment of BO
from a polynomially increasing pattern to a sawtooth pattern that has a
non-increasing trend without sacrificing convergence performance. Furthermore,
we illustrate the generalizability of the approach across two unique data sets,
two unique surrogate models, and four unique acquisition functions. All model
implementations are run on the MIT Supercloud state-of-the-art computing
hardware.