RobustGEC: Robust Grammatical Error Correction Against Subtle Context
  Perturbation
Yue Zhang Leyang Cui Enbo Zhao Wei Bi Shuming Shi
Grammatical Error Correction (GEC) systems play a vital role in assisting
people with their daily writing tasks. However, users may sometimes come across
a GEC system that initially performs well but fails to correct errors when the
inputs are slightly modified. To ensure an ideal user experience, a reliable
GEC system should have the ability to provide consistent and accurate
suggestions when encountering irrelevant context perturbations, which we refer
to as context robustness. In this paper, we introduce RobustGEC, a benchmark
designed to evaluate the context robustness of GEC systems. RobustGEC comprises
5,000 GEC cases, each with one original error-correct sentence pair and five
variants carefully devised by human annotators. Utilizing RobustGEC, we reveal
that state-of-the-art GEC systems still lack sufficient robustness against
context perturbations. In addition, we propose a simple yet effective method
for remitting this issue.