Decentralized Blockchain-based Robust Multi-agent Multi-armed Bandit
Mengfan Xu Diego Klabjan
We study a robust multi-agent multi-armed bandit problem where multiple
clients or participants are distributed on a fully decentralized blockchain,
with the possibility of some being malicious. The rewards of arms are
homogeneous among the clients, following time-invariant stochastic
distributions that are revealed to the participants only when the system is
secure enough. The system's objective is to efficiently ensure the cumulative
rewards gained by the honest participants. To this end and to the best of our
knowledge, we are the first to incorporate advanced techniques from
blockchains, as well as novel mechanisms, into the system to design optimal
strategies for honest participants. This allows various malicious behaviors and
the maintenance of participant privacy. More specifically, we randomly select a
pool of validators who have access to all participants, design a brand-new
consensus mechanism based on digital signatures for these validators, invent a
UCB-based strategy that requires less information from participants through
secure multi-party computation, and design the chain-participant interaction
and an incentive mechanism to encourage participants' participation. Notably,
we are the first to prove the theoretical guarantee of the proposed algorithms
by regret analyses in the context of optimality in blockchains. Unlike existing
work that integrates blockchains with learning problems such as federated
learning which mainly focuses on numerical optimality, we demonstrate that the
regret of honest participants is upper bounded by $log{T}$. This is consistent
with the multi-agent multi-armed bandit problem without malicious participants
and the robust multi-agent multi-armed bandit problem with purely Byzantine
attacks.