gSASRec: Reducing Overconfidence in Sequential Recommendation Trained
  with Negative Sampling
Aleksandr Petrov Craig Macdonald
A large catalogue size is one of the central challenges in training
recommendation models: a large number of items makes them memory and
computationally inefficient to compute scores for all items during training,
forcing these models to deploy negative sampling. However, negative sampling
increases the proportion of positive interactions in the training data, and
therefore models trained with negative sampling tend to overestimate the
probabilities of positive interactions a phenomenon we call overconfidence.
While the absolute values of the predicted scores or probabilities are not
important for the ranking of retrieved recommendations, overconfident models
may fail to estimate nuanced differences in the top-ranked items, resulting in
degraded performance. In this paper, we show that overconfidence explains why
the popular SASRec model underperforms when compared to BERT4Rec. This is
contrary to the BERT4Rec authors explanation that the difference in performance
is due to the bi-directional attention mechanism. To mitigate overconfidence,
we propose a novel Generalised Binary Cross-Entropy Loss function (gBCE) and
theoretically prove that it can mitigate overconfidence. We further propose the
gSASRec model, an improvement over SASRec that deploys an increased number of
negatives and the gBCE loss. We show through detailed experiments on three
datasets that gSASRec does not exhibit the overconfidence problem. As a result,
gSASRec can outperform BERT4Rec (e.g. +9.47% NDCG on the MovieLens-1M dataset),
while requiring less training time (e.g. -73% training time on MovieLens-1M).
Moreover, in contrast to BERT4Rec, gSASRec is suitable for large datasets that
contain more than 1 million items.