HAUSER: Towards Holistic and Automatic Evaluation of Simile Generation
Qianyu He Yikai Zhang Jiaqing Liang Yuncheng Huang Yanghua Xiao Yunwen Chen
Similes play an imperative role in creative writing such as story and
dialogue generation. Proper evaluation metrics are like a beacon guiding the
research of simile generation (SG). However, it remains under-explored as to
what criteria should be considered, how to quantify each criterion into
metrics, and whether the metrics are effective for comprehensive, efficient,
and reliable SG evaluation. To address the issues, we establish HAUSER, a
holistic and automatic evaluation system for the SG task, which consists of
five criteria from three perspectives and automatic metrics for each criterion.
Through extensive experiments, we verify that our metrics are significantly
more correlated with human ratings from each perspective compared with prior
automatic metrics.