Not All Weights Are Created Equal: Enhancing Energy Efficiency in
  On-Device Streaming Speech Recognition
Yang Li Yuan Shangguan Yuhao Wang Liangzhen Lai Ernie Chang Changsheng Zhao Yangyang Shi Vikas Chandra
Power consumption plays an important role in on-device streaming speech
recognition, as it has a direct impact on the user experience. This study
delves into how weight parameters in speech recognition models influence the
overall power consumption of these models. We discovered that the impact of
weight parameters on power consumption varies, influenced by factors including
how often they are invoked and their placement in memory. Armed with this
insight, we developed design guidelines aimed at optimizing on-device speech
recognition models. These guidelines focus on minimizing power use without
substantially affecting accuracy. Our method, which employs targeted
compression based on the varying sensitivities of weight parameters,
demonstrates superior performance compared to state-of-the-art compression
methods. It achieves a reduction in energy usage of up to 47% while maintaining
similar model accuracy and improving the real-time factor.