BEND: Bagging Deep Learning Training Based on Efficient Neural Network
  Diffusion
Jia Wei Xingjun Zhang Witold Pedrycz
Bagging has achieved great success in the field of machine learning by
integrating multiple base classifiers to build a single strong classifier to
reduce model variance. The performance improvement of bagging mainly relies on
the number and diversity of base classifiers. However, traditional deep
learning model training methods are expensive to train individually and
difficult to train multiple models with low similarity in a restricted dataset.
Recently, diffusion models, which have been tremendously successful in the
fields of imaging and vision, have been found to be effective in generating
neural network model weights and biases with diversity. We creatively propose a
Bagging deep learning training algorithm based on Efficient Neural network
Diffusion (BEND). The originality of BEND comes from the first use of a neural
network diffusion model to efficiently build base classifiers for bagging. Our
approach is simple but effective, first using multiple trained model weights
and biases as inputs to train autoencoder and latent diffusion model to realize
a diffusion model from noise to valid neural network parameters. Subsequently,
we generate several base classifiers using the trained diffusion model.
Finally, we integrate these ba se classifiers for various inference tasks using
the Bagging method. Resulting experiments on multiple models and datasets show
that our proposed BEND algorithm can consistently outperform the mean and
median accuracies of both the original trained model and the diffused model. At
the same time, new models diffused using the diffusion model have higher
diversity and lower cost than multiple models trained using traditional
methods. The BEND approach successfully introduces diffusion models into the
new deep learning training domain and provides a new paradigm for future deep
learning training and inference.