Discovering How Agents Learn Using Few Data
Iosif Sakos Antonios Varvitsiotis Georgios Piliouras
Decentralized learning algorithms are an essential tool for designing
multi-agent systems, as they enable agents to autonomously learn from their
experience and past interactions. In this work, we propose a theoretical and
algorithmic framework for real-time identification of the learning dynamics
that govern agent behavior using a short burst of a single system trajectory.
Our method identifies agent dynamics through polynomial regression, where we
compensate for limited data by incorporating side-information constraints that
capture fundamental assumptions or expectations about agent behavior. These
constraints are enforced computationally using sum-of-squares optimization,
leading to a hierarchy of increasingly better approximations of the true agent
dynamics. Extensive experiments demonstrated that our approach, using only 5
samples from a short run of a single trajectory, accurately recovers the true
dynamics across various benchmarks, including equilibrium selection and
prediction of chaotic systems up to 10 Lyapunov times. These findings suggest
that our approach has significant potential to support effective policy and
decision-making in strategic multi-agent systems.