Observation-Augmented Contextual Multi-Armed Bandits for Robotic
  Exploration with Uncertain Semantic Data
Shohei Wakayama Nisar Ahmed
For robotic decision-making under uncertainty, the balance between
exploitation and exploration of available options must be carefully taken into
account. In this study, we introduce a new variant of contextual multi-armed
bandits called observation-augmented CMABs (OA-CMABs) wherein a decision-making
agent can utilize extra outcome observations from an external information
source. CMABs model the expected option outcomes as a function of context
features and hidden parameters, which are inferred from previous option
outcomes. In OA-CMABs, external observations are also a function of context
features and thus provide additional evidence about the hidden parameters. Yet,
if an external information source is error-prone, the resulting posterior
updates can harm decision-making performance unless the presence of errors is
considered. To this end, we propose a robust Bayesian inference process for
OA-CMABs that is based on the concept of probabilistic data validation. Our
approach handles complex mixture model parameter priors and hybrid observation
likelihoods for semantic data sources, allowing us to develop validation
algorithms based on recently develop probabilistic semantic data association
techniques. Furthermore, to more effectively cope with the combined sources of
uncertainty in OA-CMABs, we derive a new active inference algorithm for option
selection based on expected free energy minimization. This generalizes previous
work on active inference for bandit-based robotic decision-making by accounting
for faulty observations and non-Gaussian inference. Our approaches are
demonstrated on a simulated asynchronous search site selection problem for
space exploration. The results show that even if incorrect observations are
provided by external information sources, efficient decision-making and robust
parameter inference are still achieved in a wide variety of experimental
conditions.