Identifying Important Sensory Feedback for Learning Locomotion Skills
Wanming Yu Chuanyu Yang Christopher McGreavy Eleftherios Triantafyllidis Guillaume Bellegarda Milad Shafiee Auke Jan Ijspeert Zhibin Li
Robot motor skills can be learned through deep reinforcement learning (DRL)
by neural networks as state-action mappings. While the selection of state
observations is crucial, there has been a lack of quantitative analysis to
date. Here, we present a systematic saliency analysis that quantitatively
evaluates the relative importance of different feedback states for motor skills
learned through DRL. Our approach can identify the most essential feedback
states for locomotion skills, including balance recovery, trotting, bounding,
pacing and galloping. By using only key states including joint positions,
gravity vector, base linear and angular velocities, we demonstrate that a
simulated quadruped robot can achieve robust performance in various test
scenarios across these distinct skills. The benchmarks using task performance
metrics show that locomotion skills learned with key states can achieve
comparable performance to those with all states, and the task performance or
learning success rate will drop significantly if key states are missing. This
work provides quantitative insights into the relationship between state
observations and specific types of motor skills, serving as a guideline for
robot motor learning. The proposed method is applicable to differentiable
state-action mapping, such as neural network based control policies, enabling
the learning of a wide range of motor skills with minimal sensing dependencies.