Experimental demonstration of a robust training method for strongly
  defective neuromorphic hardware
William A. Borders Advait Madhavan Matthew W. Daniels Vasileia Georgiou Martin Lueker-Boden Tiffany S. Santos Patrick M. Braganca Mark D. Stiles Jabez J. McClelland Brian D. Hoskins
The increasing scale of neural networks needed to support more complex
applications has led to an increasing requirement for area- and
energy-efficient hardware. One route to meeting the budget for these
applications is to circumvent the von Neumann bottleneck by performing
computation in or near memory. An inevitability of transferring neural networks
onto hardware is that non-idealities such as device-to-device variations or
poor device yield impact performance. Methods such as hardware-aware training,
where substrate non-idealities are incorporated during network training, are
one way to recover performance at the cost of solution generality. In this
work, we demonstrate inference on hardware neural networks consisting of 20,000
magnetic tunnel junction arrays integrated on a complementary
metal-oxide-semiconductor chips that closely resembles market-ready spin
transfer-torque magnetoresistive random access memory technology. Using 36
dies, each containing a crossbar array with its own non-idealities, we show
that even a small number of defects in physically mapped networks significantly
degrades the performance of networks trained without defects and show that, at
the cost of generality, hardware-aware training accounting for specific defects
on each die can recover to comparable performance with ideal networks. We then
demonstrate a robust training method that extends hardware-aware training to
statistics-aware training, producing network weights that perform well on most
defective dies regardless of their specific defect locations. When evaluated on
the 36 physical dies, statistics-aware trained solutions can achieve a mean
misclassification error on the MNIST dataset that differs from the
software-baseline by only 2 %. This statistics-aware training method could be
generalized to networks with many layers that are mapped to hardware suited for
industry-ready applications.