Utility-Based Reinforcement Learning: Unifying Single-objective and
  Multi-objective Reinforcement Learning
Peter Vamplew Cameron Foale Conor F. Hayes Patrick Mannion Enda Howley Richard Dazeley Scott Johnson Johan Källström Gabriel Ramos Roxana Rădulescu Willem Röpke Diederik M. Roijers
Research in multi-objective reinforcement learning (MORL) has introduced the
utility-based paradigm, which makes use of both environmental rewards and a
function that defines the utility derived by the user from those rewards. In
this paper we extend this paradigm to the context of single-objective
reinforcement learning (RL), and outline multiple potential benefits including
the ability to perform multi-policy learning across tasks relating to uncertain
objectives, risk-aware RL, discounting, and safe RL. We also examine the
algorithmic implications of adopting a utility-based approach.