Integrating Human Expertise in Continuous Spaces: A Novel Interactive
  Bayesian Optimization Framework with Preference Expected Improvement
Nikolaus Feith Elmar Rueckert
Interactive Machine Learning (IML) seeks to integrate human expertise into
machine learning processes. However, most existing algorithms cannot be applied
to Realworld Scenarios because their state spaces and/or action spaces are
limited to discrete values. Furthermore, the interaction of all existing
methods is restricted to deciding between multiple proposals. We therefore
propose a novel framework based on Bayesian Optimization (BO). Interactive
Bayesian Optimization (IBO) enables collaboration between machine learning
algorithms and humans. This framework captures user preferences and provides an
interface for users to shape the strategy by hand. Additionally, we've
incorporated a new acquisition function, Preference Expected Improvement (PEI),
to refine the system's efficiency using a probabilistic model of the user
preferences. Our approach is geared towards ensuring that machines can benefit
from human expertise, aiming for a more aligned and effective learning process.
In the course of this work, we applied our method to simulations and in a real
world task using a Franka Panda robot to show human-robot collaboration.