To Err Is Human, but Llamas Can Learn It Too
Agnes Luhtaru Taido Purason Martin Vainikko Maksym Del Mark Fishel
This study explores enhancing grammatical error correction (GEC) through
artificial error generation (AEG) using language models (LMs). Specifically, we
fine-tune Llama 2-based LMs for error generation and find that this approach
yields synthetic errors akin to human errors. Next, we train GEC Llama models
with the help of these artificial errors and outperform previous
state-of-the-art error correction models, with gains ranging between 0.8 and 6
F0.5 points across all tested languages (German, Ukrainian, and Estonian).
Moreover, we demonstrate that generating errors by fine-tuning smaller
sequence-to-sequence models and prompting large commercial LMs (GPT-3.5 and
GPT-4) also results in synthetic errors beneficially affecting error generation
models.