Open Ad Hoc Teamwork with Cooperative Game Theory
Jianhong Wang Yang Li Yuan Zhang Wei Pan Samuel Kaski
Ad hoc teamwork poses a challenging problem, requiring the design of an agent
to collaborate with teammates without prior coordination or joint training.
Open ad hoc teamwork further complicates this challenge by considering
environments with a changing number of teammates, referred to as open teams.
The state-of-the-art solution to this problem is graph-based policy learning
(GPL), leveraging the generalizability of graph neural networks to handle an
unrestricted number of agents and effectively address open teams. GPL's
performance is superior to other methods, but its joint Q-value representation
presents challenges for interpretation, hindering further development of this
research line and applicability. In this paper, we establish a new theory to
give an interpretation for the joint Q-value representation employed in GPL,
from the perspective of cooperative game theory. Building on our theory, we
propose a novel algorithm based on GPL framework, to complement the critical
features that facilitate learning, but overlooked in GPL. Through experiments,
we demonstrate the correctness of our theory by comparing the performance of
the resulting algorithm with GPL in dynamic team compositions.