Sticker820K: Empowering Interactive Retrieval with Stickers
Sijie Zhao Yixiao Ge Zhongang Qi Lin Song Xiaohan Ding Zehua Xie Ying Shan
Stickers have become a ubiquitous part of modern-day communication, conveying
complex emotions through visual imagery. To facilitate the development of more
powerful algorithms for analyzing stickers, we propose a large-scale Chinese
sticker dataset, namely Sticker820K, which consists of 820k image-text pairs.
Each sticker has rich and high-quality textual annotations, including
descriptions, optical characters, emotional labels, and style classifications.
Although vision-language tasks in the domain of natural images have been well
studied, directly applying the those models, such as CLIP, to sticker data is
not an optimal solution due to the discrepant nature between natural and
emotive image data. Therefore, we propose StickerCLIP as a benchmark model on
the Sticker820K dataset. For the text-to-image retrieval task, our StickerCLIP
demonstrates strong superiority over the CLIP, which achieves an absolute gain
of 66.0\% in mean recall on the Sticker820K test set. Additionally, we endeavor
to extend the recently popularized LLM by means of prompt tuning, integrating
its ability for sticker retrieval and allowing users to retrieve stickers
through instructions. We validate the feasibility of this method, demonstrating
the immense potential of prompt tuning in expanding LLM abilities while not
affecting the quality of upstream tasks.