PcLast: Discovering Plannable Continuous Latent States
Anurag Koul Shivakanth Sujit Shaoru Chen Ben Evans Lili Wu Byron Xu Rajan Chari Riashat Islam Raihan Seraj Yonathan Efroni Lekan Molu Miro Dudik John Langford Alex Lamb
Goal-conditioned planning benefits from learned low-dimensional
representations of rich, high-dimensional observations. While compact latent
representations, typically learned from variational autoencoders or inverse
dynamics, enable goal-conditioned planning they ignore state affordances, thus
hampering their sample-efficient planning capabilities. In this paper, we learn
a representation that associates reachable states together for effective onward
planning. We first learn a latent representation with multi-step inverse
dynamics (to remove distracting information); and then transform this
representation to associate reachable states together in $\ell_2$ space. Our
proposals are rigorously tested in various simulation testbeds. Numerical
results in reward-based and reward-free settings show significant improvements
in sampling efficiency, and yields layered state abstractions that enable
computationally efficient hierarchical planning.