From Data-Fitting to Discovery: Interpreting the Neural Dynamics of
  Motor Control through Reinforcement Learning
Eugene R. Rush Kaushik Jayaram J. Sean Humbert
In motor neuroscience, artificial recurrent neural networks models often
complement animal studies. However, most modeling efforts are limited to
data-fitting, and the few that examine virtual embodied agents in a
reinforcement learning context, do not draw direct comparisons to their
biological counterparts. Our study addressing this gap, by uncovering
structured neural activity of a virtual robot performing legged locomotion that
directly support experimental findings of primate walking and cycling. We find
that embodied agents trained to walk exhibit smooth dynamics that avoid
tangling -- or opposing neural trajectories in neighboring neural space -- a
core principle in computational neuroscience. Specifically, across a wide suite
of gaits, the agent displays neural trajectories in the recurrent layers are
less tangled than those in the input-driven actuation layers. To better
interpret the neural separation of these elliptical-shaped trajectories, we
identify speed axes that maximizes variance of mean activity across different
forward, lateral, and rotational speed conditions.