Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised
  Feature Selection
Junjing Zheng Xinyu Zhang Yongxiang Liu Weidong Jiang Kai Huo Li Liu
In the field of unsupervised feature selection, sparse principal component
analysis (SPCA) methods have attracted more and more attention recently.
Compared to spectral-based methods, SPCA methods don't rely on the construction
of a similarity matrix and show better feature selection ability on real-world
data. The original SPCA formulates a nonconvex optimization problem. Existing
convex SPCA methods reformulate SPCA as a convex model by regarding the
reconstruction matrix as an optimization variable. However, they are lack of
constraints equivalent to the orthogonality restriction in SPCA, leading to
larger solution space. In this paper, it's proved that the optimal solution to
a convex SPCA model falls onto the Positive Semidefinite (PSD) cone. A standard
convex SPCA-based model with PSD constraint for unsupervised feature selection
is proposed. Further, a two-step fast optimization algorithm via PSD projection
is presented to solve the proposed model. Two other existing convex SPCA-based
models are also proven to have their solutions optimized on the PSD cone in
this paper. Therefore, the PSD versions of these two models are proposed to
accelerate their convergence as well. We also provide a regularization
parameter setting strategy for our proposed method. Experiments on synthetic
and real-world datasets demonstrate the effectiveness and efficiency of the
proposed methods.