Symmetry-Breaking Augmentations for Ad Hoc Teamwork
Ravi Hammond Dustin Craggs Mingyu Guo Jakob Foerster Ian Reid
In many collaborative settings, artificial intelligence (AI) agents must be
able to adapt to new teammates that use unknown or previously unobserved
strategies. While often simple for humans, this can be challenging for AI
agents. For example, if an AI agent learns to drive alongside others (a
training set) that only drive on one side of the road, it may struggle to adapt
this experience to coordinate with drivers on the opposite side, even if their
behaviours are simply flipped along the left-right symmetry. To address this we
introduce symmetry-breaking augmentations (SBA), which increases diversity in
the behaviour of training teammates by applying a symmetry-flipping operation.
By learning a best-response to the augmented set of teammates, our agent is
exposed to a wider range of behavioural conventions, improving performance when
deployed with novel teammates. We demonstrate this experimentally in two
settings, and show that our approach improves upon previous ad hoc teamwork
results in the challenging card game Hanabi. We also propose a general metric
for estimating symmetry-dependency amongst a given set of policies.