From explained variance of correlated components to PCA without
  orthogonality constraints
Marie Chavent Guy Chavent
Block Principal Component Analysis (Block PCA) of a data matrix A, where
loadings Z are determined by maximization of AZ 2 over unit norm orthogonal
loadings, is difficult to use for the design of sparse PCA by 1 regularization,
due to the difficulty of taking care of both the orthogonality constraint on
loadings and the non differentiable 1 penalty. Our objective in this paper is
to relax the orthogonality constraint on loadings by introducing new objective
functions expvar(Y) which measure the part of the variance of the data matrix A
explained by correlated components Y = AZ. So we propose first a comprehensive
study of mathematical and numerical properties of expvar(Y) for two existing
definitions Zou et al. [2006], Shen and Huang [2008] and four new definitions.
Then we show that only two of these explained variance are fit to use as
objective function in block PCA formulations for A rid of orthogonality
constraints.