L-SA: Learning Under-Explored Targets in Multi-Target Reinforcement
  Learning
Kibeom Kim Hyundo Lee Min Whoo Lee Moonheon Lee Minsu Lee Byoung-Tak Zhang
Tasks that involve interaction with various targets are called multi-target
tasks. When applying general reinforcement learning approaches for such tasks,
certain targets that are difficult to access or interact with may be neglected
throughout the course of training - a predicament we call Under-explored Target
Problem (UTP). To address this problem, we propose L-SA (Learning by adaptive
Sampling and Active querying) framework that includes adaptive sampling and
active querying. In the L-SA framework, adaptive sampling dynamically samples
targets with the highest increase of success rates at a high proportion,
resulting in curricular learning from easy to hard targets. Active querying
prompts the agent to interact more frequently with under-explored targets that
need more experience or exploration. Our experimental results on visual
navigation tasks show that the L-SA framework improves sample efficiency as
well as success rates on various multi-target tasks with UTP. Also, it is
experimentally demonstrated that the cyclic relationship between adaptive
sampling and active querying effectively improves the sample richness of
under-explored targets and alleviates UTP.