Sharing the Cost of Success: A Game for Evaluating and Learning
  Collaborative Multi-Agent Instruction Giving and Following Policies
Philipp Sadler Sherzod Hakimov David Schlangen
In collaborative goal-oriented settings, the participants are not only
interested in achieving a successful outcome, but do also implicitly negotiate
the effort they put into the interaction (by adapting to each other). In this
work, we propose a challenging interactive reference game that requires two
players to coordinate on vision and language observations. The learning signal
in this game is a score (given after playing) that takes into account the
achieved goal and the players' assumed efforts during the interaction. We show
that a standard Proximal Policy Optimization (PPO) setup achieves a high
success rate when bootstrapped with heuristic partner behaviors that implement
insights from the analysis of human-human interactions. And we find that a
pairing of neural partners indeed reduces the measured joint effort when
playing together repeatedly. However, we observe that in comparison to a
reasonable heuristic pairing there is still room for improvement -- which
invites further research in the direction of cost-sharing in collaborative
interactions.