A Novel Neural Network-Based Federated Learning System for Imbalanced
  and Non-IID Data
Mahfuzur Rahman Chowdhury Muhammad Ibrahim
With the growth of machine learning techniques, privacy of data of users has
become a major concern. Most of the machine learning algorithms rely heavily on
large amount of data which may be collected from various sources. Collecting
these data yet maintaining privacy policies has become one of the most
challenging tasks for the researchers. To combat this issue, researchers have
introduced federated learning, where a prediction model is learnt by ensuring
the privacy of data of clients data. However, the prevalent federated learning
algorithms possess an accuracy and efficiency trade-off, especially for non-IID
data. In this research, we propose a centralized, neural network-based
federated learning system. The centralized algorithm incorporates micro-level
parallel processing inspired by the traditional mini-batch algorithm where the
client devices and the server handle the forward and backward propagation
respectively. We also devise a semi-centralized version of our proposed
algorithm. This algorithm takes advantage of edge computing for minimizing the
load from the central server, where clients handle both the forward and
backward propagation while sacrificing the overall train time to some extent.
We evaluate our proposed systems on five well-known benchmark datasets and
achieve satisfactory performance in a reasonable time across various data
distribution settings as compared to some existing benchmark algorithms.