Increasing Entropy to Boost Policy Gradient Performance on
  Personalization Tasks
Andrew Starnes Anton Dereventsov Clayton Webster
In this effort, we consider the impact of regularization on the diversity of
actions taken by policies generated from reinforcement learning agents trained
using a policy gradient. Policy gradient agents are prone to entropy collapse,
which means certain actions are seldomly, if ever, selected. We augment the
optimization objective function for the policy with terms constructed from
various $\varphi$-divergences and Maximum Mean Discrepancy which encourages
current policies to follow different state visitation and/or action choice
distribution than previously computed policies. We provide numerical
experiments using MNIST, CIFAR10, and Spotify datasets. The results demonstrate
the advantage of diversity-promoting policy regularization and that its use on
gradient-based approaches have significantly improved performance on a variety
of personalization tasks. Furthermore, numerical evidence is given to show that
policy regularization increases performance without losing accuracy.