Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation
  with the GeNTE Corpus
Andrea Piergentili Beatrice Savoldi Dennis Fucci Matteo Negri Luisa Bentivogli
Gender inequality is embedded in our communication practices and perpetuated
in translation technologies. This becomes particularly apparent when
translating into grammatical gender languages, where machine translation (MT)
often defaults to masculine and stereotypical representations by making undue
binary gender assumptions. Our work addresses the rising demand for inclusive
language by focusing head-on on gender-neutral translation from English to
Italian. We start from the essentials: proposing a dedicated benchmark and
exploring automated evaluation methods. First, we introduce GeNTE, a natural,
bilingual test set for gender-neutral translation, whose creation was informed
by a survey on the perception and use of neutral language. Based on GeNTE, we
then overview existing reference-based evaluation approaches, highlight their
limits, and propose a reference-free method more suitable to assess
gender-neutral translation.