Expert-guided Bayesian Optimisation for Human-in-the-loop Experimental
  Design of Known Systems
Tom Savage Ehecatl Antonio del Rio Chanona
Domain experts often possess valuable physical insights that are overlooked
in fully automated decision-making processes such as Bayesian optimisation. In
this article we apply high-throughput (batch) Bayesian optimisation alongside
anthropological decision theory to enable domain experts to influence the
selection of optimal experiments. Our methodology exploits the hypothesis that
humans are better at making discrete choices than continuous ones and enables
experts to influence critical early decisions. At each iteration we solve an
augmented multi-objective optimisation problem across a number of alternate
solutions, maximising both the sum of their utility function values and the
determinant of their covariance matrix, equivalent to their total variability.
By taking the solution at the knee point of the Pareto front, we return a set
of alternate solutions at each iteration that have both high utility values and
are reasonably distinct, from which the expert selects one for evaluation. We
demonstrate that even in the case of an uninformed practitioner, our algorithm
recovers the regret of standard Bayesian optimisation.