Goal Space Abstraction in Hierarchical Reinforcement Learning via
  Reachability Analysis
Mehdi Zadem Sergio Mover Sao Mai Nguyen
Open-ended learning benefits immensely from the use of symbolic methods for
goal representation as they offer ways to structure knowledge for efficient and
transferable learning. However, the existing Hierarchical Reinforcement
Learning (HRL) approaches relying on symbolic reasoning are often limited as
they require a manual goal representation. The challenge in autonomously
discovering a symbolic goal representation is that it must preserve critical
information, such as the environment dynamics. In this work, we propose a
developmental mechanism for subgoal discovery via an emergent representation
that abstracts (i.e., groups together) sets of environment states that have
similar roles in the task. We create a HRL algorithm that gradually learns this
representation along with the policies and evaluate it on navigation tasks to
show the learned representation is interpretable and results in data
efficiency.