Shortcutting Cross-Validation: Efficiently Deriving Column-Wise Centered
  and Scaled Training Set $\mathbf{X}^\mathbf{T}\mathbf{X}$ and
  $\mathbf{X}^\mathbf{T}\mathbf{Y}$ Without Full Recomputation of Matrix
  Products or Statistical Moments
Ole-Christian Galbo Engstrøm
Cross-validation is a widely used technique for assessing the performance of
predictive models on unseen data. Many predictive models, such as Kernel-Based
Partial Least-Squares (PLS) models, require the computation of
$\mathbf{X}^{\mathbf{T}}\mathbf{X}$ and $\mathbf{X}^{\mathbf{T}}\mathbf{Y}$
using only training set samples from the input and output matrices,
$\mathbf{X}$ and $\mathbf{Y}$, respectively. In this work, we present three
algorithms that efficiently compute these matrices. The first one allows no
column-wise preprocessing. The second one allows column-wise centering around
the training set means. The third one allows column-wise centering and
column-wise scaling around the training set means and standard deviations.
Demonstrating correctness and superior computational complexity, they offer
significant cross-validation speedup compared with straight-forward
cross-validation and previous work on fast cross-validation - all without data
leakage. Their suitability for parallelization is highlighted with an
open-source Python implementation combining our algorithms with Improved Kernel
PLS.