Tensor Network-Constrained Kernel Machines as Gaussian Processes
Frederiek Wesel Kim Batselier
Tensor Networks (TNs) have recently been used to speed up kernel machines by
constraining the model weights, yielding exponential computational and storage
savings. In this paper we prove that the outputs of Canonical Polyadic
Decomposition (CPD) and Tensor Train (TT)-constrained kernel machines recover a
Gaussian Process (GP), which we fully characterize, when placing i.i.d. priors
over their parameters. We analyze the convergence of both CPD and
TT-constrained models, and show how TT yields models exhibiting more GP
behavior compared to CPD, for the same number of model parameters. We
empirically observe this behavior in two numerical experiments where we
respectively analyze the convergence to the GP and the performance at
prediction. We thereby establish a connection between TN-constrained kernel
machines and GPs.