Federated Empirical Risk Minimization via Second-Order Method
Song Bian Zhao Song Junze Yin
Many convex optimization problems with important applications in machine
learning are formulated as empirical risk minimization (ERM). There are several
examples: linear and logistic regression, LASSO, kernel regression, quantile
regression, $p$-norm regression, support vector machines (SVM), and mean-field
variational inference. To improve data privacy, federated learning is proposed
in machine learning as a framework for training deep learning models on the
network edge without sharing data between participating nodes. In this work, we
present an interior point method (IPM) to solve a general ERM problem under the
federated learning setting. We show that the communication complexity of each
iteration of our IPM is $\tilde{O}(d^{3/2})$, where $d$ is the dimension (i.e.,
number of features) of the dataset.