Accelerating Matrix Factorization by Dynamic Pruning for Fast
  Recommendation
Yining Wu Shengyu Duan Gaole Sai Chenhong Cao Guobing Zou
Matrix factorization (MF) is a widely used collaborative filtering (CF)
algorithm for recommendation systems (RSs), due to its high prediction
accuracy, great flexibility and high efficiency in big data processing.
However, with the dramatically increased number of users/items in current RSs,
the computational complexity for training a MF model largely increases. Many
existing works have accelerated MF, by either putting in additional
computational resources or utilizing parallel systems, introducing a large
cost. In this paper, we propose algorithmic methods to accelerate MF, without
inducing any additional computational resources. In specific, we observe
fine-grained structured sparsity in the decomposed feature matrices when
considering a certain threshold. The fine-grained structured sparsity causes a
large amount of unnecessary operations during both matrix multiplication and
latent factor update, increasing the computational time of the MF training
process. Based on the observation, we firstly propose to rearrange the feature
matrices based on joint sparsity, which potentially makes a latent vector with
a smaller index more dense than that with a larger index. The feature matrix
rearrangement is given to limit the error caused by the later performed pruning
process. We then propose to prune the insignificant latent factors by an early
stopping process during both matrix multiplication and latent factor update.
The pruning process is dynamically performed according to the sparsity of the
latent factors for different users/items, to accelerate the process. The
experiments show that our method can achieve 1.2-1.65 speedups, with up to
20.08% error increase, compared with the conventional MF training process. We
also prove the proposed methods are applicable considering different
hyperparameters including optimizer, optimization strategy and initialization
method.