Limited Communications Distributed Optimization via Deep Unfolded
  Distributed ADMM
Yoav Noah Nir Shlezinger
Distributed optimization is a fundamental framework for collaborative
inference and decision making in decentralized multi-agent systems. The
operation is modeled as the joint minimization of a shared objective which
typically depends on observations gathered locally by each agent. Distributed
optimization algorithms, such as the common D-ADMM, tackle this task by
iteratively combining local computations and message exchanges. One of the main
challenges associated with distributed optimization, and particularly with
D-ADMM, is that it requires a large number of communications, i.e., messages
exchanged between the agents, to reach consensus. This can make D-ADMM costly
in power, latency, and channel resources. In this work we propose unfolded
D-ADMM, which follows the emerging deep unfolding methodology to enable D-ADMM
to operate reliably with a predefined and small number of messages exchanged by
each agent. Unfolded D-ADMM fully preserves the operation of D-ADMM, while
leveraging data to tune the hyperparameters of each iteration of the algorithm.
These hyperparameters can either be agent-specific, aiming at achieving the
best performance within a fixed number of iterations over a given network, or
shared among the agents, allowing to learn to distributedly optimize over
different networks. For both settings, our unfolded D-ADMM operates with
limited communications, while preserving the interpretability and flexibility
of the original D-ADMM algorithm. We specialize unfolded D-ADMM for two
representative settings: a distributed estimation task, considering a sparse
recovery setup, and a distributed learning scenario, where multiple agents
collaborate in learning a machine learning model. Our numerical results
demonstrate that the proposed approach dramatically reduces the number of
communications utilized by D-ADMM, without compromising on its performance.