BRNES: Enabling Security and Privacy-aware Experience Sharing in
  Multiagent Robotic and Autonomous Systems
Md Tamjid Hossain Hung Manh La Shahriar Badsha Anton Netchaev
Although experience sharing (ES) accelerates multiagent reinforcement
learning (MARL) in an advisor-advisee framework, attempts to apply ES to
decentralized multiagent systems have so far relied on trusted environments and
overlooked the possibility of adversarial manipulation and inference.
Nevertheless, in a real-world setting, some Byzantine attackers, disguised as
advisors, may provide false advice to the advisee and catastrophically degrade
the overall learning performance. Also, an inference attacker, disguised as an
advisee, may conduct several queries to infer the advisors' private information
and make the entire ES process questionable in terms of privacy leakage. To
address and tackle these issues, we propose a novel MARL framework (BRNES) that
heuristically selects a dynamic neighbor zone for each advisee at each learning
step and adopts a weighted experience aggregation technique to reduce Byzantine
attack impact. Furthermore, to keep the agent's private information safe from
adversarial inference attacks, we leverage the local differential privacy
(LDP)-induced noise during the ES process. Our experiments show that our
framework outperforms the state-of-the-art in terms of the steps to goal,
obtained reward, and time to goal metrics. Particularly, our evaluation shows
that the proposed framework is 8.32x faster than the current non-private
frameworks and 1.41x faster than the private frameworks in an adversarial
setting.