Addressing the Rank Degeneration in Sequential Recommendation via
  Singular Spectrum Smoothing
Ziwei Fan Zhiwei Liu Hao Peng Philip S. Yu
Sequential recommendation (SR) investigates the dynamic user preferences
modeling and generates the next-item prediction. The next item preference is
typically generated by the affinity between the sequence and item
representations. However, both sequence and item representations suffer from
the rank degeneration issue due to the data sparsity problem. The rank
degeneration issue significantly impairs the representations for SR. This
motivates us to measure how severe is the rank degeneration issue and alleviate
the sequence and item representation rank degeneration issues simultaneously
for SR.
  In this work, we theoretically connect the sequence representation
degeneration issue with the item rank degeneration, particularly for short
sequences and cold items. We also identify the connection between the fast
singular value decay phenomenon and the rank collapse issue in transformer
sequence output and item embeddings. We propose the area under the singular
value curve metric to evaluate the severity of the singular value decay
phenomenon and use it as an indicator of rank degeneration. We further
introduce a novel singular spectrum smoothing regularization to alleviate the
rank degeneration on both sequence and item sides, which is the Singular
sPectrum sMoothing for sequential Recommendation (SPMRec). We also establish a
correlation between the ranks of sequence and item embeddings and the rank of
the user-item preference prediction matrix, which can affect recommendation
diversity. We conduct experiments on four benchmark datasets to demonstrate the
superiority of SPMRec over the state-of-the-art recommendation methods,
especially in short sequences. The experiments also demonstrate a strong
connection between our proposed singular spectrum smoothing and recommendation
diversity.