Efficient Monaural Speech Enhancement using Spectrum Attention Fusion
Jinyu Long Jetic Gū Binhao Bai Zhibo Yang Ping Wei Junli Li
Speech enhancement is a demanding task in automated speech processing
pipelines, focusing on separating clean speech from noisy channels. Transformer
based models have recently bested RNN and CNN models in speech enhancement,
however at the same time they are much more computationally expensive and
require much more high quality training data, which is always hard to come by.
In this paper, we present an improvement for speech enhancement models that
maintains the expressiveness of self-attention while significantly reducing
model complexity, which we have termed Spectrum Attention Fusion. We carefully
construct a convolutional module to replace several self-attention layers in a
speech Transformer, allowing the model to more efficiently fuse spectral
features. Our proposed model is able to achieve comparable or better results
against SOTA models but with significantly smaller parameters (0.58M) on the
Voice Bank + DEMAND dataset.