Dynamic Transformers Provide a False Sense of Efficiency
Yiming Chen Simin Chen Zexin Li Wei Yang Cong Liu Robby T. Tan Haizhou Li
Despite much success in natural language processing (NLP), pre-trained
language models typically lead to a high computational cost during inference.
Multi-exit is a mainstream approach to address this issue by making a trade-off
between efficiency and accuracy, where the saving of computation comes from an
early exit. However, whether such saving from early-exiting is robust remains
unknown. Motivated by this, we first show that directly adapting existing
adversarial attack approaches targeting model accuracy cannot significantly
reduce inference efficiency. To this end, we propose a simple yet effective
attacking framework, SAME, a novel slowdown attack framework on multi-exit
models, which is specially tailored to reduce the efficiency of the multi-exit
models. By leveraging the multi-exit models' design characteristics, we utilize
all internal predictions to guide the adversarial sample generation instead of
merely considering the final prediction. Experiments on the GLUE benchmark show
that SAME can effectively diminish the efficiency gain of various multi-exit
models by 80% on average, convincingly validating its effectiveness and
generalization ability.