Personalized Online Federated Learning with Multiple Kernels
Pouya M. Ghari Yanning Shen
Multi-kernel learning (MKL) exhibits well-documented performance in online
non-linear function approximation. Federated learning enables a group of
learners (called clients) to train an MKL model on the data distributed among
clients to perform online non-linear function approximation. There are some
challenges in online federated MKL that need to be addressed: i) Communication
efficiency especially when a large number of kernels are considered ii)
Heterogeneous data distribution among clients. The present paper develops an
algorithmic framework to enable clients to communicate with the server to send
their updates with affordable communication cost while clients employ a large
dictionary of kernels. Utilizing random feature (RF) approximation, the present
paper proposes scalable online federated MKL algorithm. We prove that using the
proposed online federated MKL algorithm, each client enjoys sub-linear regret
with respect to the RF approximation of its best kernel in hindsight, which
indicates that the proposed algorithm can effectively deal with heterogeneity
of the data distributed among clients. Experimental results on real datasets
showcase the advantages of the proposed algorithm compared with other online
federated kernel learning ones.