GEE! Grammar Error Explanation with Large Language Models
Yixiao Song Kalpesh Krishna Rajesh Bhatt Kevin Gimpel Mohit Iyyer
Grammatical error correction tools are effective at correcting grammatical
errors in users' input sentences but do not provide users with \textit{natural
language} explanations about their errors. Such explanations are essential for
helping users learn the language by gaining a deeper understanding of its
grammatical rules (DeKeyser, 2003; Ellis et al., 2006). To address this gap, we
propose the task of grammar error explanation, where a system needs to provide
one-sentence explanations for each grammatical error in a pair of erroneous and
corrected sentences. We analyze the capability of GPT-4 in grammar error
explanation, and find that it only produces explanations for 60.2% of the
errors using one-shot prompting. To improve upon this performance, we develop a
two-step pipeline that leverages fine-tuned and prompted large language models
to perform structured atomic token edit extraction, followed by prompting GPT-4
to generate explanations. We evaluate our pipeline on German and Chinese
grammar error correction data sampled from language learners with a wide range
of proficiency levels. Human evaluation reveals that our pipeline produces
93.9% and 98.0% correct explanations for German and Chinese data, respectively.
To encourage further research in this area, we will open-source our data and
code.