SPAR: Personalized Content-Based Recommendation via Long Engagement
  Attention
Chiyu Zhang Yifei Sun Jun Chen Jie Lei Muhammad Abdul-Mageed Sinong Wang Rong Jin Sem Park Ning Yao Bo Long
Leveraging users' long engagement histories is essential for personalized
content recommendations. The success of pretrained language models (PLMs) in
NLP has led to their use in encoding user histories and candidate items,
framing content recommendations as textual semantic matching tasks. However,
existing works still struggle with processing very long user historical text
and insufficient user-item interaction. In this paper, we introduce a
content-based recommendation framework, SPAR, which effectively tackles the
challenges of holistic user interest extraction from the long user engagement
history. It achieves so by leveraging PLM, poly-attention layers and attention
sparsity mechanisms to encode user's history in a session-based manner. The
user and item side features are sufficiently fused for engagement prediction
while maintaining standalone representations for both sides, which is efficient
for practical model deployment. Moreover, we enhance user profiling by
exploiting large language model (LLM) to extract global interests from user
engagement history. Extensive experiments on two benchmark datasets demonstrate
that our framework outperforms existing state-of-the-art (SoTA) methods.