PIMbot: Policy and Incentive Manipulation for Multi-Robot Reinforcement
  Learning in Social Dilemmas
Shahab Nikkhoo Zexin Li Aritra Samanta Yufei Li Cong Liu
Recent research has demonstrated the potential of reinforcement learning (RL)
in enabling effective multi-robot collaboration, particularly in social
dilemmas where robots face a trade-off between self-interests and collective
benefits. However, environmental factors such as miscommunication and
adversarial robots can impact cooperation, making it crucial to explore how
multi-robot communication can be manipulated to achieve different outcomes.
This paper presents a novel approach, namely PIMbot, to manipulating the reward
function in multi-robot collaboration through two distinct forms of
manipulation: policy and incentive manipulation. Our work introduces a new
angle for manipulation in recent multi-agent RL social dilemmas that utilize a
unique reward function for incentivization. By utilizing our proposed PIMbot
mechanisms, a robot is able to manipulate the social dilemma environment
effectively. PIMbot has the potential for both positive and negative impacts on
the task outcome, where positive impacts lead to faster convergence to the
global optimum and maximized rewards for any chosen robot. Conversely, negative
impacts can have a detrimental effect on the overall task performance. We
present comprehensive experimental results that demonstrate the effectiveness
of our proposed methods in the Gazebo-simulated multi-robot environment. Our
work provides insights into how inter-robot communication can be manipulated
and has implications for various robotic applications. %, including robotics,
transportation, and manufacturing.