This paper proposes a new model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix data set. Our model is based on deep autoencoder with 6 layers and is trained end-to-end without any layer-wise pre-training. We empirically demonstrate that: a) deep autoencoder models generalize much better than the shallow ones, b) non-linear activation functions with negative parts are crucial for training deep models, and c) heavy use of regularization techniques such as dropout is necessary to prevent over-fitting. We also propose a new training algorithm based on iterative output re-feeding to overcome natural sparseness of collaborate filtering. The new algorithm significantly speeds up training and improves model performance. Our code is publicly available. Sites like Amazon, Netflix and Spotify use recommender systems to suggest items to users. Recommender systems can be divided into two categories: context-based and personalized recommendations.Context based recommendations take into account contextual factors such as location, date and time BID0 . Personalized recommendations typically suggest items to users using the collaborative filtering (CF) approach. In this approach the user's interests are predicted based on the analysis of tastes and preference of other users in the system and implicitly inferring "similarity" between them. The underlying assumption is that two people who have similar tastes, have a higher likelihood of having the same opinion on an item than two randomly chosen people.In designing recommender systems, the goal is to improve the accuracy of predictions. The Netflix Prize contest provides the most famous example of this problem BID1 : Netflix held the Netflix Prize to substantially improve the accuracy of the algorithm to predict user ratings for films. This is a classic CF problem: Infer the missing entries in an mxn matrix, R, whose (i, j) entry describes the ratings given by the ith user to the jth item. The performance is then measured using Root Mean Squared Error (RMSE).Training very deep autoencoders is non trivial both from optimization and regularization points of view. Early works on training auto-enocoders adapted layer-wise pre-training to solve optimization issues BID5 . In this work , we empirically show that optimization difficulties of training deep autoencoders can be solved by using scaled exponential linear units (SELUs) BID9 . This enables training without any layer-wise pre-training or residual connections. Since publicly available data sets for CF are relatively small, sufficiently large models can easily overfit. To prevent overfitting we employ heavy dropout with drop probability as high as 0.8. We also introduce a new output re-feeding training algorithm which helps to bypass the natural sparseness of updates in collaborative filtering and helps to further improve the model performance. Deep learning has revolutionized many areas of machine learning, and it is poised do so with recommender systems as well. In this paper we demonstrated how very deep autoencoders can be successfully trained even on relatively small amounts of data by using both well established (dropout) and relatively recent ("scaled exponential linear units") deep learning techniques. Further, we introduced iterative output re-feeding -a technique which allowed us to perform dense updates in collaborative filtering, increase learning rate and further improve generalization performance of our model. On the task of future rating prediction, our model outperforms other approaches even without using additional temporal signals.While our code supports item-based model (such as I-AutoRec) we argue that this approach is less practical than user-based model (U-AutoRec). This is because in real-world recommender systems, there are usually much more users then items. Finally, when building personalized recommender system and faced with scaling problems, it can be acceptable to sample items but not users.
This paper demonstrates how to train deep autoencoders end-to-end to achieve SoA results on time-split Netflix data set. This paper presents a deep autoencoder model for rating prediction that outperforms other state-of-the-art approahces on the Netflix prize dataset.  Proposes to use a deep AE to do rating prediction tasks in recommender systems. The authors present a model for more accurate Netflix recommendations demonstrating that a deep autoencoder can out-perform more complex RNN-based models that have temporal information. 