Global Convergence of Policy Gradient Methods in Reinforcement Learning,
  Games and Control
Shicong Cen Yuejie Chi
Policy gradient methods, where one searches for the policy of interest by
maximizing the value functions using first-order information, become
increasingly popular for sequential decision making in reinforcement learning,
games, and control. Guaranteeing the global optimality of policy gradient
methods, however, is highly nontrivial due to nonconcavity of the value
functions. In this exposition, we highlight recent progresses in understanding
and developing policy gradient methods with global convergence guarantees,
putting an emphasis on their finite-time convergence rates with regard to
salient problem parameters.