Examining the Role and Limits of Batchnorm Optimization to Mitigate
  Diverse Hardware-noise in In-memory Computing
Abhiroop Bhattacharjee Abhishek Moitra Youngeun Kim Yeshwanth Venkatesha Priyadarshini Panda
In-Memory Computing (IMC) platforms such as analog crossbars are gaining
focus as they facilitate the acceleration of low-precision Deep Neural Networks
(DNNs) with high area- & compute-efficiencies. However, the intrinsic
non-idealities in crossbars, which are often non-deterministic and non-linear,
degrade the performance of the deployed DNNs. In addition to quantization
errors, most frequently encountered non-idealities during inference include
crossbar circuit-level parasitic resistances and device-level non-idealities
such as stochastic read noise and temporal drift. In this work, our goal is to
closely examine the distortions caused by these non-idealities on the
dot-product operations in analog crossbars and explore the feasibility of a
nearly training-less solution via crossbar-aware fine-tuning of batchnorm
parameters in real-time to mitigate the impact of the non-idealities. This
enables reduction in hardware costs in terms of memory and training energy for
IMC noise-aware retraining of the DNN weights on crossbars.