On Minimum Trace Factor Analysis -- An Old Song Sung to a New Tune
C. Li A. Shkolnik
Dimensionality reduction methods, such as principal component analysis (PCA)
and factor analysis, are central to many problems in data science. There are,
however, serious and well-understood challenges to finding robust low
dimensional approximations for data with significant heteroskedastic noise.
This paper introduces a relaxed version of Minimum Trace Factor Analysis
(MTFA), a convex optimization method with roots dating back to the work of
Ledermann in 1940. This relaxation is particularly effective at not overfitting
to heteroskedastic perturbations and addresses the commonly cited Heywood cases
in factor analysis and the recently identified "curse of ill-conditioning" for
existing spectral methods. We provide theoretical guarantees on the accuracy of
the resulting low rank subspace and the convergence rate of the proposed
algorithm to compute that matrix. We develop a number of interesting
connections to existing methods, including HeteroPCA, Lasso, and Soft-Impute,
to fill an important gap in the already large literature on low rank matrix
estimation. Numerical experiments benchmark our results against several recent
proposals for dealing with heteroskedastic noise.