F-PABEE: Flexible-patience-based Early Exiting for Single-label and
  Multi-label text Classification Tasks
Xiangxiang Gao Wei Zhu Jiasheng Gao Congrui Yin
Computational complexity and overthinking problems have become the
bottlenecks for pre-training language models (PLMs) with millions or even
trillions of parameters. A Flexible-Patience-Based Early Exiting method
(F-PABEE) has been proposed to alleviate the problems mentioned above for
single-label classification (SLC) and multi-label classification (MLC) tasks.
F-PABEE makes predictions at the classifier and will exit early if predicted
distributions of cross-layer are consecutively similar. It is more flexible
than the previous state-of-the-art (SOTA) early exiting method PABEE because it
can simultaneously adjust the similarity score thresholds and the patience
parameters. Extensive experiments show that: (1) F-PABEE makes a better
speedup-accuracy balance than existing early exiting strategies on both SLC and
MLC tasks. (2) F-PABEE achieves faster inference and better performances on
different PLMs such as BERT and ALBERT. (3) F-PABEE-JSKD performs best for
F-PABEE with different similarity measures.