Performance Analysis of DNN Inference/Training with Convolution and
  non-Convolution Operations
Hadi Esmaeilzadeh Soroush Ghodrati Andrew B. Kahng Sean Kinzer Susmita Dey Manasi Sachin S. Sapatnekar Zhiang Wang
Today's performance analysis frameworks for deep learning accelerators suffer
from two significant limitations. First, although modern convolutional neural
network (CNNs) consist of many types of layers other than convolution,
especially during training, these frameworks largely focus on convolution
layers only. Second, these frameworks are generally targeted towards inference,
and lack support for training operations. This work proposes a novel
performance analysis framework, SimDIT, for general ASIC-based systolic
hardware accelerator platforms. The modeling effort of SimDIT comprehensively
covers convolution and non-convolution operations of both CNN inference and
training on a highly parameterizable hardware substrate. SimDIT is integrated
with a backend silicon implementation flow and provides detailed end-to-end
performance statistics (i.e., data access cost, cycle counts, energy, and
power) for executing CNN inference and training workloads. SimDIT-enabled
performance analysis reveals that on a 64X64 processing array, non-convolution
operations constitute 59.5% of total runtime for ResNet-50 training workload.
In addition, by optimally distributing available off-chip DRAM bandwidth and
on-chip SRAM resources, SimDIT achieves 18X performance improvement over a
generic static resource allocation for ResNet-50 inference.