To the Max: Reinventing Reward in Reinforcement Learning
Grigorii Veviurko Wendelin Böhmer Mathijs de Weerdt
In reinforcement learning (RL), different rewards can define the same optimal
policy but result in drastically different learning performance. For some, the
agent gets stuck with a suboptimal behavior, and for others, it solves the task
efficiently. Choosing a good reward function is hence an extremely important
yet challenging problem. In this paper, we explore an alternative approach to
using rewards for learning. We introduce max-reward RL, where an agent
optimizes the maximum rather than the cumulative reward. Unlike earlier works,
our approach works for deterministic and stochastic environments and can be
easily combined with state-of-the-art RL algorithms. In the experiments, we
study the performance of max-reward RL algorithms in two goal-reaching
environments from Gymnasium-Robotics and demonstrate its benefits over standard
RL. The code is publicly available.