FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with
  Human Feedback
Ashish Singh Prateek Agarwal Zixuan Huang Arpita Singh Tong Yu Sungchul Kim Victor Bursztyn Nikos Vlassis Ryan A. Rossi
Captions are crucial for understanding scientific visualizations and
documents. Existing captioning methods for scientific figures rely on
figure-caption pairs extracted from documents for training, many of which fall
short with respect to metrics like helpfulness, explainability, and
visual-descriptiveness [15] leading to generated captions being misaligned with
reader preferences. To enable the generation of high-quality figure captions,
we introduce FigCaps-HF a new framework for figure-caption generation that can
incorporate domain expert feedback in generating captions optimized for reader
preferences. Our framework comprises of 1) an automatic method for evaluating
quality of figure-caption pairs, 2) a novel reinforcement learning with human
feedback (RLHF) method to optimize a generative figure-to-caption model for
reader preferences. We demonstrate the effectiveness of our simple learning
framework by improving performance over standard fine-tuning across different
types of models. In particular, when using BLIP as the base model, our RLHF
framework achieves a mean gain of 35.7%, 16.9%, and 9% in ROUGE, BLEU, and
Meteor, respectively. Finally, we release a large-scale benchmark dataset with
human feedback on figure-caption pairs to enable further evaluation and
development of RLHF techniques for this problem.