Diverse Policies Converge in Reward-free Markov Decision Processe
Fanqi Lin Shiyu Huang Weiwei Tu
Reinforcement learning has achieved great success in many decision-making
tasks, and traditional reinforcement learning algorithms are mainly designed
for obtaining a single optimal solution. However, recent works show the
importance of developing diverse policies, which makes it an emerging research
topic. Despite the variety of diversity reinforcement learning algorithms that
have emerged, none of them theoretically answer the question of how the
algorithm converges and how efficient the algorithm is. In this paper, we
provide a unified diversity reinforcement learning framework and investigate
the convergence of training diverse policies. Under such a framework, we also
propose a provably efficient diversity reinforcement learning algorithm.
Finally, we verify the effectiveness of our method through numerical
experiments.