Diagnosing and exploiting the computational demands of videos games for
  deep reinforcement learning
Lakshmi Narasimhan Govindarajan Rex G Liu Drew Linsley Alekh Karkada Ashok Max Reuter Michael J Frank Thomas Serre
Humans learn by interacting with their environments and perceiving the
outcomes of their actions. A landmark in artificial intelligence has been the
development of deep reinforcement learning (dRL) algorithms capable of doing
the same in video games, on par with or better than humans. However, it remains
unclear whether the successes of dRL models reflect advances in visual
representation learning, the effectiveness of reinforcement learning algorithms
at discovering better policies, or both. To address this question, we introduce
the Learning Challenge Diagnosticator (LCD), a tool that separately measures
the perceptual and reinforcement learning demands of a task. We use LCD to
discover a novel taxonomy of challenges in the Procgen benchmark, and
demonstrate that these predictions are both highly reliable and can instruct
algorithmic development. More broadly, the LCD reveals multiple failure cases
that can occur when optimizing dRL algorithms over entire video game benchmarks
like Procgen, and provides a pathway towards more efficient progress.