Holonic Learning: A Flexible Agent-based Distributed Machine Learning
  Framework
Ahmad Esmaeili Zahra Ghorrati Eric T. Matson
Ever-increasing ubiquity of data and computational resources in the last
decade have propelled a notable transition in the machine learning paradigm
towards more distributed approaches. Such a transition seeks to not only tackle
the scalability and resource distribution challenges but also to address
pressing privacy and security concerns. To contribute to the ongoing discourse,
this paper introduces Holonic Learning (HoL), a collaborative and
privacy-focused learning framework designed for training deep learning models.
By leveraging holonic concepts, the HoL framework establishes a structured
self-similar hierarchy in the learning process, enabling more nuanced control
over collaborations through the individual model aggregation approach of each
holon, along with their intra-holon commitment and communication patterns. HoL,
in its general form, provides extensive design and flexibility potentials. For
empirical analysis and to demonstrate its effectiveness, this paper implements
HoloAvg, a special variant of HoL that employs weighted averaging for model
aggregation across all holons. The convergence of the proposed method is
validated through experiments on both IID and Non-IID settings of the standard
MNISt dataset. Furthermore, the performance behaviors of HoL are investigated
under various holarchical designs and data distribution scenarios. The
presented results affirm HoL's prowess in delivering competitive performance
particularly, in the context of the Non-IID data distribution.