Federated Learning for Sparse Principal Component Analysis
Sin Cheng Ciou Pin Jui Chen Elvin Y. Tseng Yuh-Jye Lee
In the rapidly evolving realm of machine learning, algorithm effectiveness
often faces limitations due to data quality and availability. Traditional
approaches grapple with data sharing due to legal and privacy concerns. The
federated learning framework addresses this challenge. Federated learning is a
decentralized approach where model training occurs on client sides, preserving
privacy by keeping data localized. Instead of sending raw data to a central
server, only model updates are exchanged, enhancing data security. We apply
this framework to Sparse Principal Component Analysis (SPCA) in this work. SPCA
aims to attain sparse component loadings while maximizing data variance for
improved interpretability. Beside the L1 norm regularization term in
conventional SPCA, we add a smoothing function to facilitate gradient-based
optimization methods. Moreover, in order to improve computational efficiency,
we introduce a least squares approximation to original SPCA. This enables
analytic solutions on the optimization processes, leading to substantial
computational improvements. Within the federated framework, we formulate SPCA
as a consensus optimization problem, which can be solved using the Alternating
Direction Method of Multipliers (ADMM). Our extensive experiments involve both
IID and non-IID random features across various data owners. Results on
synthetic and public datasets affirm the efficacy of our federated SPCA
approach.