Gradable ChatGPT Translation Evaluation
Hui Jiao Bei Peng Lu Zong Xiaojun Zhang Xinwei Li
ChatGPT, as a language model based on large-scale pre-training, has exerted a
profound influence on the domain of machine translation. In ChatGPT, a "Prompt"
refers to a segment of text or instruction employed to steer the model towards
generating a specific category of response. The design of the translation
prompt emerges as a key aspect that can wield influence over factors such as
the style, precision and accuracy of the translation to a certain extent.
However, there is a lack of a common standard and methodology on how to design
and select a translation prompt. Accordingly, this paper proposes a generic
taxonomy, which defines gradable translation prompts in terms of expression
type, translation style, POS information and explicit statement, thus
facilitating the construction of prompts endowed with distinct attributes
tailored for various translation tasks. Specific experiments and cases are
selected to validate and illustrate the effectiveness of the method.