Full-Stack Optimization for CAM-Only DNN Inference
João Paulo C. de Lima Asif Ali Khan Luigi Carro Jeronimo Castrillon
The accuracy of neural networks has greatly improved across various domains
over the past years. Their ever-increasing complexity, however, leads to
prohibitively high energy demands and latency in von Neumann systems. Several
computing-in-memory (CIM) systems have recently been proposed to overcome this,
but trade-offs involving accuracy, hardware reliability, and scalability for
large models remain a challenge. Additionally, for some CIM designs, the
activation movement still requires considerable time and energy. This paper
explores the combination of algorithmic optimizations for ternary weight neural
networks and associative processors (APs) implemented using racetrack memory
(RTM). We propose a novel compilation flow to optimize convolutions on APs by
reducing their arithmetic intensity. By leveraging the benefits of RTM-based
APs, this approach substantially reduces data transfers within the memory while
addressing accuracy, energy efficiency, and reliability concerns. Concretely,
our solution improves the energy efficiency of ResNet-18 inference on ImageNet
by 7.5x compared to crossbar in-memory accelerators while retaining software
accuracy.