BREATHE: Second-Order Gradients and Heteroscedastic Emulation based
  Design Space Exploration
Shikhar Tuli Niraj K. Jha
Researchers constantly strive to explore larger and more complex search
spaces in various scientific studies and physical experiments. However, such
investigations often involve sophisticated simulators or time-consuming
experiments that make exploring and observing new design samples challenging.
Previous works that target such applications are typically sample-inefficient
and restricted to vector search spaces. To address these limitations, this work
proposes a constrained multi-objective optimization (MOO) framework, called
BREATHE, that searches not only traditional vector-based design spaces but also
graph-based design spaces to obtain best-performing graphs. It leverages
second-order gradients and actively trains a heteroscedastic surrogate model
for sample-efficient optimization. In a single-objective vector optimization
application, it leads to 64.1% higher performance than the next-best baseline,
random forest regression. In graph-based search, BREATHE outperforms the
next-best baseline, i.e., a graphical version of Gaussian-process-based
Bayesian optimization, with up to 64.9% higher performance. In a MOO task, it
achieves up to 21.9$\times$ higher hypervolume than the state-of-the-art
method, multi-objective Bayesian optimization (MOBOpt). BREATHE also
outperforms the baseline methods on most standard MOO benchmark applications.