Robust Driving Policy Learning with Guided Meta Reinforcement Learning
Kanghoon Lee Jiachen Li David Isele Jinkyoo Park Kikuo Fujimura Mykel J. Kochenderfer
Although deep reinforcement learning (DRL) has shown promising results for
autonomous navigation in interactive traffic scenarios, existing work typically
adopts a fixed behavior policy to control social vehicles in the training
environment. This may cause the learned driving policy to overfit the
environment, making it difficult to interact well with vehicles with different,
unseen behaviors. In this work, we introduce an efficient method to train
diverse driving policies for social vehicles as a single meta-policy. By
randomizing the interaction-based reward functions of social vehicles, we can
generate diverse objectives and efficiently train the meta-policy through
guiding policies that achieve specific objectives. We further propose a
training strategy to enhance the robustness of the ego vehicle's driving policy
using the environment where social vehicles are controlled by the learned
meta-policy. Our method successfully learns an ego driving policy that
generalizes well to unseen situations with out-of-distribution (OOD) social
agents' behaviors in a challenging uncontrolled T-intersection scenario.