RACE-IT: A Reconfigurable Analog CAM-Crossbar Engine for In-Memory
  Transformer Acceleration
Lei Zhao Luca Buonanno Ron M. Roth Sergey Serebryakov Archit Gajjar John Moon Jim Ignowski Giacomo Pedretti
Transformer models represent the cutting edge of Deep Neural Networks (DNNs)
and excel in a wide range of machine learning tasks. However, processing these
models demands significant computational resources and results in a substantial
memory footprint. While In-memory Computing (IMC) offers promise for
accelerating Matrix-Vector Multiplications (MVMs) with high computational
parallelism and minimal data movement, employing it for implementing other
crucial operators within DNNs remains a formidable task. This challenge is
exacerbated by the extensive use of Softmax and data-dependent matrix
multiplications within the attention mechanism. Furthermore, existing IMC
designs encounter difficulties in fully harnessing the benefits of analog MVM
acceleration due to the area and energy-intensive nature of Analog-to-Digital
Converters (ADCs). To tackle these challenges, we introduce a novel Compute
Analog Content Addressable Memory (Compute-ACAM) structure capable of
performing various non-MVM operations within Transformers. Together with the
crossbar structure, our proposed RACE-IT accelerator enables efficient
execution of all operations within Transformer models in the analog domain.
Given the flexibility of our proposed Compute-ACAMs to perform arbitrary
operations, RACE-IT exhibits adaptability to diverse non-traditional and future
DNN architectures without necessitating hardware modifications. Leveraging the
capability of Compute-ACAMs to process analog input and produce digital output,
we also replace ADCs, thereby reducing the overall area and energy costs. By
evaluating various Transformer models against state-of-the-art GPUs and
existing IMC accelerators, RACE-IT increases performance by 10.7x and 5.9x, and
reduces energy by 1193x, and 3.9x, respectively