Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning
Shuo Tang Rui Ye Chenxin Xu Xiaowen Dong Siheng Chen Yanfeng Wang
Decentralized and lifelong-adaptive multi-agent collaborative learning aims
to enhance collaboration among multiple agents without a central server, with
each agent solving varied tasks over time. To achieve efficient collaboration,
agents should: i) autonomously identify beneficial collaborative relationships
in a decentralized manner; and ii) adapt to dynamically changing task
observations. In this paper, we propose DeLAMA, a decentralized multi-agent
lifelong collaborative learning algorithm with dynamic collaboration graphs. To
promote autonomous collaboration relationship learning, we propose a
decentralized graph structure learning algorithm, eliminating the need for
external priors. To facilitate adaptation to dynamic tasks, we design a memory
unit to capture the agents' accumulated learning history and knowledge, while
preserving finite storage consumption. To further augment the system's
expressive capabilities and computational efficiency, we apply algorithm
unrolling, leveraging the advantages of both mathematical optimization and
neural networks. This allows the agents to `learn to collaborate' through the
supervision of training tasks. Our theoretical analysis verifies that
inter-agent collaboration is communication efficient under a small number of
communication rounds. The experimental results verify its ability to facilitate
the discovery of collaboration strategies and adaptation to dynamic learning
scenarios, achieving a 98.80% reduction in MSE and a 188.87% improvement in
classification accuracy. We expect our work can serve as a foundational
technique to facilitate future works towards an intelligent, decentralized, and
dynamic multi-agent system. Code is available at
https://github.com/ShuoTang123/DeLAMA.