A Lexical-aware Non-autoregressive Transformer-based ASR Model
Chong-En Lin Kuan-Yu Chen
Non-autoregressive automatic speech recognition (ASR) has become a mainstream
of ASR modeling because of its fast decoding speed and satisfactory result. To
further boost the performance, relaxing the conditional independence assumption
and cascading large-scaled pre-trained models are two active research
directions. In addition to these strategies, we propose a lexical-aware
non-autoregressive Transformer-based (LA-NAT) ASR framework, which consists of
an acoustic encoder, a speech-text shared encoder, and a speech-text shared
decoder. The acoustic encoder is used to process the input speech features as
usual, and the speech-text shared encoder and decoder are designed to train
speech and text data simultaneously. By doing so, LA-NAT aims to make the ASR
model aware of lexical information, so the resulting model is expected to
achieve better results by leveraging the learned linguistic knowledge. A series
of experiments are conducted on the AISHELL-1, CSJ, and TEDLIUM 2 datasets.
According to the experiments, the proposed LA-NAT can provide superior results
than other recently proposed non-autoregressive ASR models. In addition, LA-NAT
is a relatively compact model than most non-autoregressive ASR models, and it
is about 58 times faster than the classic autoregressive model.