Hyperparameter Optimization for Multi-Objective Reinforcement Learning
Florian Felten Daniel Gareev El-Ghazali Talbi Grégoire Danoy
Reinforcement learning (RL) has emerged as a powerful approach for tackling
complex problems. The recent introduction of multi-objective reinforcement
learning (MORL) has further expanded the scope of RL by enabling agents to make
trade-offs among multiple objectives. This advancement not only has broadened
the range of problems that can be tackled but also created numerous
opportunities for exploration and advancement. Yet, the effectiveness of RL
agents heavily relies on appropriately setting their hyperparameters. In
practice, this task often proves to be challenging, leading to unsuccessful
deployments of these techniques in various instances. Hence, prior research has
explored hyperparameter optimization in RL to address this concern.
  This paper presents an initial investigation into the challenge of
hyperparameter optimization specifically for MORL. We formalize the problem,
highlight its distinctive challenges, and propose a systematic methodology to
address it. The proposed methodology is applied to a well-known environment
using a state-of-the-art MORL algorithm, and preliminary results are reported.
Our findings indicate that the proposed methodology can effectively provide
hyperparameter configurations that significantly enhance the performance of
MORL agents. Furthermore, this study identifies various future research
opportunities to further advance the field of hyperparameter optimization for
MORL.