DyCE: Dynamic Configurable Exiting for Deep Learning Compression and
  Scaling
Qingyuan Wang Barry Cardiff Antoine Frappé Benoit Larras Deepu John
Modern deep learning (DL) models necessitate the employment of scaling and
compression techniques for effective deployment in resource-constrained
environments. Most existing techniques, such as pruning and quantization are
generally static. On the other hand, dynamic compression methods, such as early
exits, reduce complexity by recognizing the difficulty of input samples and
allocating computation as needed. Dynamic methods, despite their superior
flexibility and potential for co-existing with static methods, pose significant
challenges in terms of implementation due to any changes in dynamic parts will
influence subsequent processes. Moreover, most current dynamic compression
designs are monolithic and tightly integrated with base models, thereby
complicating the adaptation to novel base models. This paper introduces DyCE,
an dynamic configurable early-exit framework that decouples design
considerations from each other and from the base model. Utilizing this
framework, various types and positions of exits can be organized according to
predefined configurations, which can be dynamically switched in real-time to
accommodate evolving performance-complexity requirements. We also propose
techniques for generating optimized configurations based on any desired
trade-off between performance and computational complexity. This empowers
future researchers to focus on the improvement of individual exits without
latent compromise of overall system performance. The efficacy of this approach
is demonstrated through image classification tasks with deep CNNs. DyCE
significantly reduces the computational complexity by 23.5% of ResNet152 and
25.9% of ConvNextv2-tiny on ImageNet, with accuracy reductions of less than
0.5%. Furthermore, DyCE offers advantages over existing dynamic methods in
terms of real-time configuration and fine-grained performance tuning.