Agile and versatile bipedal robot tracking control through reinforcement
  learning
Jiayi Li Linqi Ye Yi Cheng Houde Liu Bin Liang
The remarkable athletic intelligence displayed by humans in complex dynamic
movements such as dancing and gymnastics suggests that the balance mechanism in
biological beings is decoupled from specific movement patterns. This decoupling
allows for the execution of both learned and unlearned movements under certain
constraints while maintaining balance through minor whole-body coordination. To
replicate this balance ability and body agility, this paper proposes a
versatile controller for bipedal robots. This controller achieves ankle and
body trajectory tracking across a wide range of gaits using a single
small-scale neural network, which is based on a model-based IK solver and
reinforcement learning. We consider a single step as the smallest control unit
and design a universally applicable control input form suitable for any
single-step variation. Highly flexible gait control can be achieved by
combining these minimal control units with high-level policy through our
extensible control interface. To enhance the trajectory-tracking capability of
our controller, we utilize a three-stage training curriculum. After training,
the robot can move freely between target footholds at varying distances and
heights. The robot can also maintain static balance without repeated stepping
to adjust posture. Finally, we evaluate the tracking accuracy of our controller
on various bipedal tasks, and the effectiveness of our control framework is
verified in the simulation environment.