BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning
Jing Cui Yufei Han Yuzhe Ma Jianbin Jiao Junge Zhang
Backdoor attacks in reinforcement learning (RL) have previously employed
intense attack strategies to ensure attack success. However, these methods
suffer from high attack costs and increased detectability. In this work, we
propose a novel approach, BadRL, which focuses on conducting highly sparse
backdoor poisoning efforts during training and testing while maintaining
successful attacks. Our algorithm, BadRL, strategically chooses state
observations with high attack values to inject triggers during training and
testing, thereby reducing the chances of detection. In contrast to the previous
methods that utilize sample-agnostic trigger patterns, BadRL dynamically
generates distinct trigger patterns based on targeted state observations,
thereby enhancing its effectiveness. Theoretical analysis shows that the
targeted backdoor attack is always viable and remains stealthy under specific
assumptions. Empirical results on various classic RL tasks illustrate that
BadRL can substantially degrade the performance of a victim agent with minimal
poisoning efforts 0.003% of total training steps) during training and
infrequent attacks during testing.