Attention Weighted Mixture of Experts with Contrastive Learning for
  Personalized Ranking in E-commerce
Juan Gong Zhenlin Chen Chaoyi Ma Zhuojian Xiao Haonan Wang Guoyu Tang Lin Liu Sulong Xu Bo Long Yunjiang Jiang
Ranking model plays an essential role in e-commerce search and
recommendation. An effective ranking model should give a personalized ranking
list for each user according to the user preference. Existing algorithms
usually extract a user representation vector from the user behavior sequence,
then feed the vector into a feed-forward network (FFN) together with other
features for feature interactions, and finally produce a personalized ranking
score. Despite tremendous progress in the past, there is still room for
improvement. Firstly, the personalized patterns of feature interactions for
different users are not explicitly modeled. Secondly, most of existing
algorithms have poor personalized ranking results for long-tail users with few
historical behaviors due to the data sparsity. To overcome the two challenges,
we propose Attention Weighted Mixture of Experts (AW-MoE) with contrastive
learning for personalized ranking. Firstly, AW-MoE leverages the MoE framework
to capture personalized feature interactions for different users. To model the
user preference, the user behavior sequence is simultaneously fed into expert
networks and the gate network. Within the gate network, one gate unit and one
activation unit are designed to adaptively learn the fine-grained activation
vector for experts using an attention mechanism. Secondly, a random masking
strategy is applied to the user behavior sequence to simulate long-tail users,
and an auxiliary contrastive loss is imposed to the output of the gate network
to improve the model generalization for these users. This is validated by a
higher performance gain on the long-tail user test set. Experiment results on a
JD real production dataset and a public dataset demonstrate the effectiveness
of AW-MoE, which significantly outperforms state-of-art methods. Notably,
AW-MoE has been successfully deployed in the JD e-commerce search engine, ...