FedLion: Faster Adaptive Federated Optimization with Fewer Communication
Zhiwei Tang Tsung-Hui Chang
In Federated Learning (FL), a framework to train machine learning models
across distributed data, well-known algorithms like FedAvg tend to have slow
convergence rates, resulting in high communication costs during training. To
address this challenge, we introduce FedLion, an adaptive federated
optimization algorithm that seamlessly incorporates key elements from the
recently proposed centralized adaptive algorithm, Lion (Chen et al. 2o23), into
the FL framework. Through comprehensive evaluations on two widely adopted FL
benchmarks, we demonstrate that FedLion outperforms previous state-of-the-art
adaptive algorithms, including FAFED (Wu et al. 2023) and FedDA. Moreover,
thanks to the use of signed gradients in local training, FedLion substantially
reduces data transmission requirements during uplink communication when
compared to existing adaptive algorithms, further reducing communication costs.
Last but not least, this work also includes a novel theoretical analysis,
showcasing that FedLion attains faster convergence rate than established FL
algorithms like FedAvg.