How Robust Are Energy-Based Models Trained With Equilibrium Propagation?
Siddharth Mansingh Michal Kucer Garrett Kenyon Juston Moore Michael Teti
Deep neural networks (DNNs) are easily fooled by adversarial perturbations
that are imperceptible to humans. Adversarial training, a process where
adversarial examples are added to the training set, is the current
state-of-the-art defense against adversarial attacks, but it lowers the model's
accuracy on clean inputs, is computationally expensive, and offers less
robustness to natural noise. In contrast, energy-based models (EBMs), which
were designed for efficient implementation in neuromorphic hardware and
physical systems, incorporate feedback connections from each layer to the
previous layer, yielding a recurrent, deep-attractor architecture which we
hypothesize should make them naturally robust. Our work is the first to explore
the robustness of EBMs to both natural corruptions and adversarial attacks,
which we do using the CIFAR-10 and CIFAR-100 datasets. We demonstrate that EBMs
are more robust than transformers and display comparable robustness to
adversarially-trained DNNs on gradient-based (white-box) attacks, query-based
(black-box) attacks, and natural perturbations without sacrificing clean
accuracy, and without the need for adversarial training or additional training
techniques.