Federated Distributionally Robust Optimization with Non-Convex
  Objectives: Algorithm and Analysis
Yang Jiao Kai Yang Dongjin Song
Distributionally Robust Optimization (DRO), which aims to find an optimal
decision that minimizes the worst case cost over the ambiguity set of
probability distribution, has been widely applied in diverse applications,
e.g., network behavior analysis, risk management, etc. However, existing DRO
techniques face three key challenges: 1) how to deal with the asynchronous
updating in a distributed environment; 2) how to leverage the prior
distribution effectively; 3) how to properly adjust the degree of robustness
according to different scenarios. To this end, we propose an asynchronous
distributed algorithm, named Asynchronous Single-looP alternatIve gRadient
projEction (ASPIRE) algorithm with the itErative Active SEt method (EASE) to
tackle the federated distributionally robust optimization (FDRO) problem.
Furthermore, a new uncertainty set, i.e., constrained D-norm uncertainty set,
is developed to effectively leverage the prior distribution and flexibly
control the degree of robustness. Finally, our theoretical analysis elucidates
that the proposed algorithm is guaranteed to converge and the iteration
complexity is also analyzed. Extensive empirical studies on real-world datasets
demonstrate that the proposed method can not only achieve fast convergence, and
remain robust against data heterogeneity as well as malicious attacks, but also
tradeoff robustness with performance.