Over-the-Air Federated Learning with Compressed Sensing: Is
  Sparsification Necessary?
Adrian Edin Zheng Chen
Over-the-Air (OtA) Federated Learning (FL) refers to an FL system where
multiple agents apply OtA computation for transmitting model updates to a
common edge server. Two important features of OtA computation, namely linear
processing and signal-level superposition, motivate the use of linear
compression with compressed sensing (CS) methods to reduce the number of data
samples transmitted over the channel. The previous works on applying CS methods
in OtA FL have primarily assumed that the original model update vectors are
sparse, or they have been sparsified before compression. However, it is unclear
whether linear compression with CS-based reconstruction is more effective than
directly sending the non-zero elements in the sparsified update vectors, under
the same total power constraint. In this study, we examine and compare several
communication designs with or without sparsification. Our findings demonstrate
that sparsification before compression is not necessary. Alternatively,
sparsification without linear compression can also achieve better performance
than the commonly considered setup that combines both.