CGCE: A Chinese Generative Chat Evaluation Benchmark for General and
  Financial Domains
Xuanyu Zhang Bingbing Li Qing Yang
Generative chat models, such as ChatGPT and GPT-4, have revolutionized
natural language generation (NLG) by incorporating instructions and human
feedback to achieve significant performance improvements. However, the lack of
standardized evaluation benchmarks for chat models, particularly for Chinese
and domain-specific models, hinders their assessment and progress. To address
this gap, we introduce the Chinese Generative Chat Evaluation (CGCE) benchmark,
focusing on general and financial domains. The CGCE benchmark encompasses
diverse tasks, including 200 questions in the general domain and 150 specific
professional questions in the financial domain. Manual scoring evaluates
factors such as accuracy, coherence, expression clarity, and completeness. The
CGCE benchmark provides researchers with a standardized framework to assess and
compare Chinese generative chat models, fostering advancements in NLG research.