Advancing IIoT with Over-the-Air Federated Learning: The Role of
  Iterative Magnitude Pruning
Fazal Muhammad Ali Khan Hatem Abou-Zeid Aryan Kaushik Syed Ali Hassan
The industrial Internet of Things (IIoT) under Industry 4.0 heralds an era of
interconnected smart devices where data-driven insights and machine learning
(ML) fuse to revolutionize manufacturing. A noteworthy development in IIoT is
the integration of federated learning (FL), which addresses data privacy and
security among devices. FL enables edge sensors, also known as peripheral
intelligence units (PIUs) to learn and adapt using their data locally, without
explicit sharing of confidential data, to facilitate a collaborative yet
confidential learning process. However, the lower memory footprint and
computational power of PIUs inherently require deep neural network (DNN) models
that have a very compact size. Model compression techniques such as pruning can
be used to reduce the size of DNN models by removing unnecessary connections
that have little impact on the model's performance, thus making the models more
suitable for the limited resources of PIUs. Targeting the notion of compact yet
robust DNN models, we propose the integration of iterative magnitude pruning
(IMP) of the DNN model being trained in an over-the-air FL (OTA-FL) environment
for IIoT. We provide a tutorial overview and also present a case study of the
effectiveness of IMP in OTA-FL for an IIoT environment. Finally, we present
future directions for enhancing and optimizing these deep compression
techniques further, aiming to push the boundaries of IIoT capabilities in
acquiring compact yet robust and high-performing DNN models.