Zeroth-order Asynchronous Learning with Bounded Delays with a Use-case
  in Resource Allocation in Communication Networks
Pourya Behmandpoor Marc Moonen Panagiotis Patrinos
Distributed optimization has experienced a significant surge in interest due
to its wide-ranging applications in distributed learning and adaptation. While
various scenarios, such as shared-memory, local-memory, and consensus-based
approaches, have been extensively studied in isolation, there remains a need
for further exploration of their interconnections. This paper specifically
concentrates on a scenario where agents collaborate toward a unified mission
while potentially having distinct tasks. Each agent's actions can potentially
impact other agents through interactions. Within this context, the objective
for the agents is to optimize their local parameters based on the aggregate of
local reward functions, where only local zeroth-order oracles are available.
Notably, the learning process is asynchronous, meaning that agents update and
query their zeroth-order oracles asynchronously while communicating with other
agents subject to bounded but possibly random communication delays. This paper
presents theoretical convergence analyses and establishes a convergence rate
for the proposed approach. Furthermore, it addresses the relevant issue of deep
learning-based resource allocation in communication networks and conducts
numerical experiments in which agents, acting as transmitters, collaboratively
train their individual (possibly unique) policies to maximize a common
performance metric.