The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to
  Improve Generalization, Stability, and Privacy in Federated Learning
Elaheh Jafarigol Theodore Trafalis
In a data-centric era, concerns regarding privacy and ethical data handling
grow as machine learning relies more on personal information. This empirical
study investigates the privacy, generalization, and stability of deep learning
models in the presence of additive noise in federated learning frameworks. Our
main objective is to provide strategies to measure the generalization,
stability, and privacy-preserving capabilities of these models and further
improve them. To this end, five noise infusion mechanisms at varying noise
levels within centralized and federated learning settings are explored. As
model complexity is a key component of the generalization and stability of deep
learning models during training and evaluation, a comparative analysis of three
Convolutional Neural Network (CNN) architectures is provided. The paper
introduces Signal-to-Noise Ratio (SNR) as a quantitative measure of the
trade-off between privacy and training accuracy of noise-infused models, aiming
to find the noise level that yields optimal privacy and accuracy. Moreover, the
Price of Stability and Price of Anarchy are defined in the context of
privacy-preserving deep learning, contributing to the systematic investigation
of the noise infusion strategies to enhance privacy without compromising
performance. Our research sheds light on the delicate balance between these
critical factors, fostering a deeper understanding of the implications of
noise-based regularization in machine learning. By leveraging noise as a tool
for regularization and privacy enhancement, we aim to contribute to the
development of robust, privacy-aware algorithms, ensuring that AI-driven
solutions prioritize both utility and privacy.