Structured Low-Rank Tensors for Generalized Linear Models
Batoul Taki Anand D. Sarwate Waheed U. Bajwa
Recent works have shown that imposing tensor structures on the coefficient
tensor in regression problems can lead to more reliable parameter estimation
and lower sample complexity compared to vector-based methods. This work
investigates a new low-rank tensor model, called Low Separation Rank (LSR), in
Generalized Linear Model (GLM) problems. The LSR model -- which generalizes the
well-known Tucker and CANDECOMP/PARAFAC (CP) models, and is a special case of
the Block Tensor Decomposition (BTD) model -- is imposed onto the coefficient
tensor in the GLM model. This work proposes a block coordinate descent
algorithm for parameter estimation in LSR-structured tensor GLMs. Most
importantly, it derives a minimax lower bound on the error threshold on
estimating the coefficient tensor in LSR tensor GLM problems. The minimax bound
is proportional to the intrinsic degrees of freedom in the LSR tensor GLM
problem, suggesting that its sample complexity may be significantly lower than
that of vectorized GLMs. This result can also be specialised to lower bound the
estimation error in CP and Tucker-structured GLMs. The derived bounds are
comparable to tight bounds in the literature for Tucker linear regression, and
the tightness of the minimax lower bound is further assessed numerically.
Finally, numerical experiments on synthetic datasets demonstrate the efficacy
of the proposed LSR tensor model for three regression types (linear, logistic
and Poisson). Experiments on a collection of medical imaging datasets
demonstrate the usefulness of the LSR model over other tensor models (Tucker
and CP) on real, imbalanced data with limited available samples.