Whisper in Focus: Enhancing Stuttered Speech Classification with Encoder
  Layer Optimization
Huma Ameer Seemab Latif Rabia Latif Sana Mukhtar
In recent years, advancements in the field of speech processing have led to
cutting-edge deep learning algorithms with immense potential for real-world
applications. The automated identification of stuttered speech is one of such
applications that the researchers are addressing by employing deep learning
techniques. Recently, researchers have utilized Wav2vec2.0, a speech
recognition model to classify disfluency types in stuttered speech. Although
Wav2vec2.0 has shown commendable results, its ability to generalize across all
disfluency types is limited. In addition, since its base model uses 12 encoder
layers, it is considered a resource-intensive model. Our study unravels the
capabilities of Whisper for the classification of disfluency types in stuttered
speech. We have made notable contributions in three pivotal areas: enhancing
the quality of SEP28-k benchmark dataset, exploration of Whisper for
classification, and introducing an efficient encoder layer freezing strategy.
The optimized Whisper model has achieved the average F1-score of 0.81, which
proffers its abilities. This study also unwinds the significance of deeper
encoder layers in the identification of disfluency types, as the results
demonstrate their greater contribution compared to initial layers. This
research represents substantial contributions, shifting the emphasis towards an
efficient solution, thereby thriving towards prospective innovation.