TERM Model: Tensor Ring Mixture Model for Density Estimation
Ruituo Wu Jiani Liu Ce Zhu Anh-Huy Phan Ivan V. Oseledets Yipeng Liu
Efficient probability density estimation is a core challenge in statistical
machine learning. Tensor-based probabilistic graph methods address
interpretability and stability concerns encountered in neural network
approaches. However, a substantial number of potential tensor permutations can
lead to a tensor network with the same structure but varying expressive
capabilities. In this paper, we take tensor ring decomposition for density
estimator, which significantly reduces the number of permutation candidates
while enhancing expressive capability compared with existing used
decompositions. Additionally, a mixture model that incorporates multiple
permutation candidates with adaptive weights is further designed, resulting in
increased expressive flexibility and comprehensiveness. Different from the
prevailing directions of tensor network structure/permutation search, our
approach provides a new viewpoint inspired by ensemble learning. This approach
acknowledges that suboptimal permutations can offer distinctive information
besides that of optimal permutations. Experiments show the superiority of the
proposed approach in estimating probability density for moderately dimensional
datasets and sampling to capture intricate details.