Bring the Noise: Introducing Noise Robustness to Pretrained Automatic
  Speech Recognition
Patrick Eickhoff Matthias Möller Theresa Pekarek Rosin Johannes Twiefel Stefan Wermter
In recent research, in the domain of speech processing, large End-to-End
(E2E) systems for Automatic Speech Recognition (ASR) have reported
state-of-the-art performance on various benchmarks. These systems intrinsically
learn how to handle and remove noise conditions from speech. Previous research
has shown, that it is possible to extract the denoising capabilities of these
models into a preprocessor network, which can be used as a frontend for
downstream ASR models. However, the proposed methods were limited to specific
fully convolutional architectures. In this work, we propose a novel method to
extract the denoising capabilities, that can be applied to any encoder-decoder
architecture. We propose the Cleancoder preprocessor architecture that extracts
hidden activations from the Conformer ASR model and feeds them to a decoder to
predict denoised spectrograms. We train our pre-processor on the Noisy Speech
Database (NSD) to reconstruct denoised spectrograms from noisy inputs. Then, we
evaluate our model as a frontend to a pretrained Conformer ASR model as well as
a frontend to train smaller Conformer ASR models from scratch. We show that the
Cleancoder is able to filter noise from speech and that it improves the total
Word Error Rate (WER) of the downstream model in noisy conditions for both
applications.