ACNPU: A 4.75TOPS/W 1080P@30FPS Super Resolution Accelerator with
  Decoupled Asymmetric Convolution
Tun-Hao Yang Tian-Sheuan Chang
Deep learning-driven superresolution (SR) outperforms traditional techniques
but also faces the challenge of high complexity and memory bandwidth. This
challenge leads many accelerators to opt for simpler and shallow models like
FSRCNN, compromising performance for real-time needs, especially for
resource-limited edge devices. This paper proposes an energy-efficient SR
accelerator, ACNPU, to tackle this challenge. The ACNPU enhances image quality
by 0.34dB with a 27-layer model, but needs 36\% less complexity than FSRCNN,
while maintaining a similar model size, with the \textit{decoupled asymmetric
convolution and split-bypass structure}. The hardware-friendly 17K-parameter
model enables \textit{holistic model fusion} instead of localized layer fusion
to remove external DRAM access of intermediate feature maps. The on-chip memory
bandwidth is further reduced with the \textit{input stationary flow} and
\textit{parallel-layer execution} to reduce power consumption. Hardware is
regular and easy to control to support different layers by \textit{processing
elements (PEs) clusters with reconfigurable input and uniform data flow}. The
implementation in the 40 nm CMOS process consumes 2333 K gate counts and 198KB
SRAMs. The ACNPU achieves 31.7 FPS and 124.4 FPS for x2 and x4 scales Full-HD
generation, respectively, which attains 4.75 TOPS/W energy efficiency.