Enabling Multi-Agent Transfer Reinforcement Learning via Scenario
  Independent Representation
Ayesha Siddika Nipu Siming Liu Anthony Harris
Multi-Agent Reinforcement Learning (MARL) algorithms are widely adopted in
tackling complex tasks that require collaboration and competition among agents
in dynamic Multi-Agent Systems (MAS). However, learning such tasks from scratch
is arduous and may not always be feasible, particularly for MASs with a large
number of interactive agents due to the extensive sample complexity. Therefore,
reusing knowledge gained from past experiences or other agents could
efficiently accelerate the learning process and upscale MARL algorithms. In
this study, we introduce a novel framework that enables transfer learning for
MARL through unifying various state spaces into fixed-size inputs that allow
one unified deep-learning policy viable in different scenarios within a MAS. We
evaluated our approach in a range of scenarios within the StarCraft Multi-Agent
Challenge (SMAC) environment, and the findings show significant enhancements in
multi-agent learning performance using maneuvering skills learned from other
scenarios compared to agents learning from scratch. Furthermore, we adopted
Curriculum Transfer Learning (CTL), enabling our deep learning policy to
progressively acquire knowledge and skills across pre-designed homogeneous
learning scenarios organized by difficulty levels. This process promotes inter-
and intra-agent knowledge transfer, leading to high multi-agent learning
performance in more complicated heterogeneous scenarios.