Federated learning compression designed for lightweight communications
Lucas Grativol Ribeiro Mathieu Leonardon Guillaume Muller Virginie Fresse Matthieu Arzel
Federated Learning (FL) is a promising distributed method for edge-level
machine learning, particularly for privacysensitive applications such as those
in military and medical domains, where client data cannot be shared or
transferred to a cloud computing server. In many use-cases, communication cost
is a major challenge in FL due to its natural intensive network usage. Client
devices, such as smartphones or Internet of Things (IoT) nodes, have limited
resources in terms of energy, computation, and memory. To address these
hardware constraints, lightweight models and compression techniques such as
pruning and quantization are commonly adopted in centralised paradigms. In this
paper, we investigate the impact of compression techniques on FL for a typical
image classification task. Going further, we demonstrate that a straightforward
method can compresses messages up to 50% while having less than 1% of accuracy
loss, competing with state-of-the-art techniques.