Adaptive Policy Learning to Additional Tasks
Wenjian Hao Zehui Lu Zihao Liang Tianyu Zhou Shaoshuai Mou
This paper develops a policy learning method for tuning a pre-trained policy
to adapt to additional tasks without altering the original task. A method named
Adaptive Policy Gradient (APG) is proposed in this paper, which combines
Bellman's principle of optimality with the policy gradient approach to improve
the convergence rate. This paper provides theoretical analysis which guarantees
the convergence rate and sample complexity of $\mathcal{O}(1/T)$ and
$\mathcal{O}(1/\epsilon)$, respectively, where $T$ denotes the number of
iterations and $\epsilon$ denotes the accuracy of the resulting stationary
policy. Furthermore, several challenging numerical simulations, including
cartpole, lunar lander, and robot arm, are provided to show that APG obtains
similar performance compared to existing deterministic policy gradient methods
while utilizing much less data and converging at a faster rate.