Machine learning with tree tensor networks, CP rank constraints, and
  tensor dropout
Hao Chen Thomas Barthel
Tensor networks approximate order-$N$ tensors with a reduced number of
degrees of freedom that is only polynomial in $N$ and arranged as a network of
partially contracted smaller tensors. As suggested in [arXiv:2205.15296] in the
context of quantum many-body physics, computation costs can be further
substantially reduced by imposing constraints on the canonical polyadic (CP)
rank of the tensors in such networks. Here we demonstrate how tree tensor
networks (TTN) with CP rank constraints and tensor dropout can be used in
machine learning. The approach is found to outperform other tensor-network
based methods in Fashion-MNIST image classification. A low-rank TTN classifier
with branching ratio $b=4$ reaches test set accuracy 90.3\% with low
computation costs. Consisting of mostly linear elements, tensor network
classifiers avoid the vanishing gradient problem of deep neural networks. The
CP rank constraints have additional advantages: The number of parameters can be
decreased and tuned more freely to control overfitting, improve generalization
properties, and reduce computation costs. They allow us to employ trees with
large branching ratios which substantially improves the representation power.