ViCo: Engaging Video Comment Generation with Human Preference Rewards
Yuchong Sun Bei Liu Xu Chen Ruihua Song Jianlong Fu
Engaging video comments play an important role in video social media, as they
are the carrier of feelings, thoughts, or humor of the audience. Preliminary
works have made initial exploration for video comment generation by adopting
caption-style encoder-decoder models. However, comment generation presents some
unique challenges distinct from caption generation, which makes these methods
somewhat less effective at generating engaging comments. In contrast to the
objective and descriptive nature of captions, comments tend to be inherently
subjective, making it hard to quantify and evaluate the engagement of comments.
Furthermore, the scarcity of truly engaging comments brings difficulty to
collecting enough high-quality training examples. In this paper, we propose
ViCo with three novel designs to tackle the above challenges for generating
engaging Video Comments. Firstly, to quantify the engagement of comments, we
utilize the number of "likes" each comment receives as a proxy of human
preference after an appropriate debiasing procedure. Secondly, to automatically
evaluate the engagement of comments, we train a reward model to align its
judgment to the above proxy. Our user studies indicate that this reward model
effectively aligns with human judgments. Lastly, to alleviate the scarcity of
high-quality comments, an initial generator is trained on readily available but
noisy data to generate comments. Then the reward model is employed to offer
feedback on the generated comments, thus optimizing the initial generator. To
facilitate the research of video commenting, we collect a large video
comment-dataset (ViCo-20k) with rich metadata from a popular video website.
Experiments on ViCo-20k show that the comments generated by our ViCo model
exhibit the best performance in terms of both quantitative and qualitative
results, particularly when engagement is considered.