On Quantum Natural Policy Gradients
André Sequeira Luis Paulo Santos Luis Soares Barbosa
This research delves into the role of the quantum Fisher Information Matrix
(FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based
reinforcement learning agents. While previous studies have highlighted the
effectiveness of PQC-based policies preconditioned with the quantum FIM in
contextual bandits, its impact in broader reinforcement learning contexts, such
as Markov Decision Processes, is less clear. Through a detailed analysis of
L\"owner inequalities between quantum and classical FIMs, this study uncovers
the nuanced distinctions and implications of using each type of FIM. Our
results indicate that a PQC-based agent using the quantum FIM without
additional insights typically incurs a larger approximation error and does not
guarantee improved performance compared to the classical FIM. Empirical
evaluations in classic control benchmarks suggest even though quantum FIM
preconditioning outperforms standard gradient ascent, in general it is not
superior to classical FIM preconditioning.