Towards End-to-End Spoken Grammatical Error Correction
Stefano Bannò Rao Ma Mengjie Qian Kate M. Knill Mark J. F. Gales
Grammatical feedback is crucial for L2 learners, teachers, and testers.
Spoken grammatical error correction (GEC) aims to supply feedback to L2
learners on their use of grammar when speaking. This process usually relies on
a cascaded pipeline comprising an ASR system, disfluency removal, and GEC, with
the associated concern of propagating errors between these individual modules.
In this paper, we introduce an alternative "end-to-end" approach to spoken GEC,
exploiting a speech recognition foundation model, Whisper. This foundation
model can be used to replace the whole framework or part of it, e.g., ASR and
disfluency removal. These end-to-end approaches are compared to more standard
cascaded approaches on the data obtained from a free-speaking spoken language
assessment test, Linguaskill. Results demonstrate that end-to-end spoken GEC is
possible within this architecture, but the lack of available data limits
current performance compared to a system using large quantities of text-based
GEC data. Conversely, end-to-end disfluency detection and removal, which is
easier for the attention-based Whisper to learn, does outperform cascaded
approaches. Additionally, the paper discusses the challenges of providing
feedback to candidates when using end-to-end systems for spoken GEC.