USDC: Unified Static and Dynamic Compression for Visual Transformer
Huan Yuan Chao Liao Jianchao Tan Peng Yao Jiyuan Jia Bin Chen Chengru Song Di Zhang
Visual Transformers have achieved great success in almost all vision tasks,
such as classification, detection, and so on. However, the model complexity and
the inference speed of the visual transformers hinder their deployments in
industrial products. Various model compression techniques focus on directly
compressing the visual transformers into a smaller one while maintaining the
model performance, however, the performance drops dramatically when the
compression ratio is large. Furthermore, several dynamic network techniques
have also been applied to dynamically compress the visual transformers to
obtain input-adaptive efficient sub-structures during the inference stage,
which can achieve a better trade-off between the compression ratio and the
model performance. The upper bound of memory of dynamic models is not reduced
in the practical deployment since the whole original visual transformer model
and the additional control gating modules should be loaded onto devices
together for inference. To alleviate two disadvantages of two categories of
methods, we propose to unify the static compression and dynamic compression
techniques jointly to obtain an input-adaptive compressed model, which can
further better balance the total compression ratios and the model performances.
Moreover, in practical deployment, the batch sizes of the training and
inference stage are usually different, which will cause the model inference
performance to be worse than the model training performance, which is not
touched by all previous dynamic network papers. We propose a sub-group gates
augmentation technique to solve this performance drop problem. Extensive
experiments demonstrate the superiority of our method on various baseline
visual transformers such as DeiT, T2T-ViT, and so on.