NumHG: A Dataset for Number-Focused Headline Generation
Jian-Tao Huang Chung-Chi Chen Hen-Hsen Huang Hsin-Hsi Chen
Headline generation, a key task in abstractive summarization, strives to
condense a full-length article into a succinct, single line of text. Notably,
while contemporary encoder-decoder models excel based on the ROUGE metric, they
often falter when it comes to the precise generation of numerals in headlines.
We identify the lack of datasets providing fine-grained annotations for
accurate numeral generation as a major roadblock. To address this, we introduce
a new dataset, the NumHG, and provide over 27,000 annotated numeral-rich news
articles for detailed investigation. Further, we evaluate five well-performing
models from previous headline generation tasks using human evaluation in terms
of numerical accuracy, reasonableness, and readability. Our study reveals a
need for improvement in numerical accuracy, demonstrating the potential of the
NumHG dataset to drive progress in number-focused headline generation and
stimulate further discussions in numeral-focused text generation.