Tracking Control for a Spherical Pendulum via Curriculum Reinforcement
  Learning
Pascal Klink Florian Wolf Kai Ploeger Jan Peters Joni Pajarinen
Reinforcement Learning (RL) allows learning non-trivial robot control laws
purely from data. However, many successful applications of RL have relied on
ad-hoc regularizations, such as hand-crafted curricula, to regularize the
learning performance. In this paper, we pair a recent algorithm for
automatically building curricula with RL on massively parallelized simulations
to learn a tracking controller for a spherical pendulum on a robotic arm via
RL. Through an improved optimization scheme that better respects the
non-Euclidean task structure, we allow the method to reliably generate
curricula of trajectories to be tracked, resulting in faster and more robust
learning compared to an RL baseline that does not exploit this form of
structured learning. The learned policy matches the performance of an optimal
control baseline on the real system, demonstrating the potential of curriculum
RL to jointly learn state estimation and control for non-linear tracking tasks.