Online Tensor Inference
Xin Wen Will Wei Sun Yichen Zhang
Recent technological advances have led to contemporary applications that
demand real-time processing and analysis of sequentially arriving tensor data.
Traditional offline learning, involving the storage and utilization of all data
in each computational iteration, becomes impractical for high-dimensional
tensor data due to its voluminous size. Furthermore, existing low-rank tensor
methods lack the capability for statistical inference in an online fashion,
which is essential for real-time predictions and informed decision-making. This
paper addresses these challenges by introducing a novel online inference
framework for low-rank tensor learning. Our approach employs Stochastic
Gradient Descent (SGD) to enable efficient real-time data processing without
extensive memory requirements, thereby significantly reducing computational
demands. We establish a non-asymptotic convergence result for the online
low-rank SGD estimator, nearly matches the minimax optimal rate of estimation
error in offline models that store all historical data. Building upon this
foundation, we propose a simple yet powerful online debiasing approach for
sequential statistical inference in low-rank tensor learning. The entire online
procedure, covering both estimation and inference, eliminates the need for data
splitting or storing historical data, making it suitable for on-the-fly
hypothesis testing. Given the sequential nature of our data collection,
traditional analyses relying on offline methods and sample splitting are
inadequate. In our analysis, we control the sum of constructed
super-martingales to ensure estimates along the entire solution path remain
within the benign region. Additionally, a novel spectral representation tool is
employed to address statistical dependencies among iterative estimates,
establishing the desired asymptotic normality.