COURIER: Contrastive User Intention Reconstruction for Large-Scale
  Pre-Train of Image Features
Jia-Qi Yang Chenglei Dai OU Dan Ju Huang De-Chuan Zhan Qingwen Liu Xiaoyi Zeng Yang Yang
With the development of the multi-media internet, visual characteristics have
become an important factor affecting user interests. Thus, incorporating visual
features is a promising direction for further performance improvements in
click-through rate (CTR) prediction. However, we found that simply injecting
the image embeddings trained with established pre-training methods only has
marginal improvements. We attribute the failure to two reasons: First, The
pre-training methods are designed for well-defined computer vision tasks
concentrating on semantic features, and they cannot learn personalized interest
in recommendations. Secondly, pre-trained image embeddings only containing
semantic information have little information gain, considering we already have
semantic features such as categories and item titles as inputs in the CTR
prediction task. We argue that a pre-training method tailored for
recommendation is necessary for further improvements. To this end, we propose a
recommendation-aware image pre-training method that can learn visual features
from user click histories. Specifically, we propose a user interest
reconstruction module to mine visual features related to user interests from
behavior histories. We further propose a contrastive training method to avoid
collapsing of embedding vectors. We conduct extensive experiments to verify
that our method can learn users' visual interests, and our method achieves
$0.46\%$ improvement in offline AUC and $0.88\%$ improvement in Taobao online
GMV with p-value$<0.01$.