Learning Discrete-Time Major-Minor Mean Field Games
Kai Cui Gökçe Dayanıklı Mathieu Laurière Matthieu Geist Olivier Pietquin Heinz Koeppl
Recent techniques based on Mean Field Games (MFGs) allow the scalable
analysis of multi-player games with many similar, rational agents. However,
standard MFGs remain limited to homogeneous players that weakly influence each
other, and cannot model major players that strongly influence other players,
severely limiting the class of problems that can be handled. We propose a novel
discrete time version of major-minor MFGs (M3FGs), along with a learning
algorithm based on fictitious play and partitioning the probability simplex.
Importantly, M3FGs generalize MFGs with common noise and can handle not only
random exogeneous environment states but also major players. A key challenge is
that the mean field is stochastic and not deterministic as in standard MFGs.
Our theoretical investigation verifies both the M3FG model and its algorithmic
solution, showing firstly the well-posedness of the M3FG model starting from a
finite game of interest, and secondly convergence and approximation guarantees
of the fictitious play algorithm. Then, we empirically verify the obtained
theoretical results, ablating some of the theoretical assumptions made, and
show successful equilibrium learning in three example problems. Overall, we
establish a learning framework for a novel and broad class of tractable games.