Spatial-SpinDrop: Spatial Dropout-based Binary Bayesian Neural Network
  with Spintronics Implementation
Soyed Tuhin Ahmed Kamal Danouchi Michael Hefenbrock Guillaume Prenat Lorena Anghel Mehdi B. Tahoori
Recently, machine learning systems have gained prominence in real-time,
critical decision-making domains, such as autonomous driving and industrial
automation. Their implementations should avoid overconfident predictions
through uncertainty estimation. Bayesian Neural Networks (BayNNs) are
principled methods for estimating predictive uncertainty. However, their
computational costs and power consumption hinder their widespread deployment in
edge AI. Utilizing Dropout as an approximation of the posterior distribution,
binarizing the parameters of BayNNs, and further to that implementing them in
spintronics-based computation-in-memory (CiM) hardware arrays provide can be a
viable solution. However, designing hardware Dropout modules for convolutional
neural network (CNN) topologies is challenging and expensive, as they may
require numerous Dropout modules and need to use spatial information to drop
certain elements. In this paper, we introduce MC-SpatialDropout, a spatial
dropout-based approximate BayNNs with spintronics emerging devices. Our method
utilizes the inherent stochasticity of spintronic devices for efficient
implementation of the spatial dropout module compared to existing
implementations. Furthermore, the number of dropout modules per network layer
is reduced by a factor of $9\times$ and energy consumption by a factor of
$94.11\times$, while still achieving comparable predictive performance and
uncertainty estimates compared to related works.