SAFE-GIL: SAFEty Guided Imitation Learning
Yusuf Umut Ciftci Zeyuan Feng Somil Bansal
Behavior Cloning is a popular approach to Imitation Learning, in which a
robot observes an expert supervisor and learns a control policy. However,
behavior cloning suffers from the "compounding error" problem - the policy
errors compound as it deviates from the expert demonstrations and might lead to
catastrophic system failures, limiting its use in safety-critical applications.
On-policy data aggregation methods are able to address this issue at the cost
of rolling out and repeated training of the imitation policy, which can be
tedious and computationally prohibitive. We propose SAFE-GIL, an off-policy
behavior cloning method that guides the expert via adversarial disturbance
during data collection. The algorithm abstracts the imitation error as an
adversarial disturbance in the system dynamics, injects it during data
collection to expose the expert to safety critical states, and collects
corrective actions. Our method biases training to more closely replicate expert
behavior in safety-critical states and allows more variance in less critical
states. We compare our method with several behavior cloning techniques and
DAgger on autonomous navigation and autonomous taxiing tasks and show higher
task success and safety, especially in low data regimes where the likelihood of
error is higher, at a slight drop in the performance.