Adversarial Style Transfer for Robust Policy Optimization in Deep
  Reinforcement Learning
Md Masudur Rahman Yexiang Xue
This paper proposes an algorithm that aims to improve generalization for
reinforcement learning agents by removing overfitting to confounding features.
Our approach consists of a max-min game theoretic objective. A generator
transfers the style of observation during reinforcement learning. An additional
goal of the generator is to perturb the observation, which maximizes the
agent's probability of taking a different action. In contrast, a policy network
updates its parameters to minimize the effect of such perturbations, thus
staying robust while maximizing the expected future reward. Based on this
setup, we propose a practical deep reinforcement learning algorithm,
Adversarial Robust Policy Optimization (ARPO), to find a robust policy that
generalizes to unseen environments. We evaluate our approach on Procgen and
Distracting Control Suite for generalization and sample efficiency.
Empirically, ARPO shows improved performance compared to a few baseline
algorithms, including data augmentation.