Empirical Study of Straggler Problem in Parameter Server on Iterative
  Convergent Distributed Machine Learning
Benjamin Wong
The purpose of this study is to test the effectiveness of current straggler
mitigation techniques over different important iterative convergent machine
learning(ML) algorithm including Matrix Factorization (MF), Multinomial
Logistic Regression (MLR), and Latent Dirichlet Allocation (LDA) . The
experiment was conducted to implemented using the FlexPS system, which is the
latest system implementation that employ parameter server architecture. The
experiment employed the Bulk Synchronous Parallel (BSP) computational model to
examine the straggler problem in Parameter Server on Iterative Convergent
Distributed Machine Learning. Moreover, the current research analyzes the
experimental arrangement of the parameter server strategy concerning the
parallel learning problems by injecting universal straggler patterns and
executing latest mitigation techniques. The findings of the study are
significant in that as they will provide the necessary platform for conducting
further research into the problem and allow the researcher to compare different
methods for various applications. The outcome is therefore expected to
facilitate the development of new techniques coupled with new perspectives in
addressing this problem.