Useful Compact Representations for Data-Fitting
Johannes J. Brust
For minimization problems without 2nd derivative information, methods that
estimate Hessian matrices can be very effective. However, conventional
techniques generate dense matrices that are prohibitive for large problems.
Limited-memory compact representations express the dense arrays in terms of a
low rank representation and have become the state-of-the-art for software
implementations on large deterministic problems. We develop new compact
representations that are parameterized by a choice of vectors and that reduce
to existing well known formulas for special choices. We demonstrate
effectiveness of the compact representations for large eigenvalue computations,
tensor factorizations and nonlinear regressions.