FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local
  Parameter Sharing
Yongzhe Jia Xuyun Zhang Amin Beheshti Wanchun Dou
Federated Learning (FL) has emerged as a promising solution in Edge Computing
(EC) environments to process the proliferation of data generated by edge
devices. By collaboratively optimizing the global machine learning models on
distributed edge devices, FL circumvents the need for transmitting raw data and
enhances user privacy. Despite practical successes, FL still confronts
significant challenges including constrained edge device resources, multiple
tasks deployment, and data heterogeneity. However, existing studies focus on
mitigating the FL training costs of each single task whereas neglecting the
resource consumption across multiple tasks in heterogeneous FL scenarios. In
this paper, we propose Heterogeneous Federated Learning with Local Parameter
Sharing (FedLPS) to fill this gap. FedLPS leverages principles from transfer
learning to facilitate the deployment of multiple tasks on a single device by
dividing the local model into a shareable encoder and task-specific encoders.
To further reduce resource consumption, a channel-wise model pruning algorithm
that shrinks the footprint of local models while accounting for both data and
system heterogeneity is employed in FedLPS. Additionally, a novel heterogeneous
model aggregation algorithm is proposed to aggregate the heterogeneous
predictors in FedLPS. We implemented the proposed FedLPS on a real FL platform
and compared it with state-of-the-art (SOTA) FL frameworks. The experimental
results on five popular datasets and two modern DNN models illustrate that the
proposed FedLPS significantly outperforms the SOTA FL frameworks by up to 4.88%
and reduces the computational resource consumption by 21.3%. Our code is
available at:https://github.com/jyzgh/FedLPS.