BOtied: Multi-objective Bayesian optimization with tied multivariate
  ranks
Ji Won Park Nataša Tagasovska Michael Maser Stephen Ra Kyunghyun Cho
Many scientific and industrial applications require joint optimization of
multiple, potentially competing objectives. Multi-objective Bayesian
optimization (MOBO) is a sample-efficient framework for identifying
Pareto-optimal solutions. We show a natural connection between non-dominated
solutions and the highest multivariate rank, which coincides with the outermost
level line of the joint cumulative distribution function (CDF). We propose the
CDF indicator, a Pareto-compliant metric for evaluating the quality of
approximate Pareto sets that complements the popular hypervolume indicator. At
the heart of MOBO is the acquisition function, which determines the next
candidate to evaluate by navigating the best compromises among the objectives.
Multi-objective acquisition functions that rely on box decomposition of the
objective space, such as the expected hypervolume improvement (EHVI) and
entropy search, scale poorly to a large number of objectives. We propose an
acquisition function, called BOtied, based on the CDF indicator. BOtied can be
implemented efficiently with copulas, a statistical tool for modeling complex,
high-dimensional distributions. We benchmark BOtied against common acquisition
functions, including EHVI and random scalarization (ParEGO), in a series of
synthetic and real-data experiments. BOtied performs on par with the baselines
across datasets and metrics while being computationally efficient.