Scalable CP Decomposition for Tensor Learning using GPU Tensor Cores
Zeliang Zhang Zhuo Liu Susan Liang Zhiyuan Wang Yifan Zhu Chen Ding Chenliang Xu
CP decomposition is a powerful tool for data science, especially gene
analysis, deep learning, and quantum computation. However, the application of
tensor decomposition is largely hindered by the exponential increment of the
computational complexity and storage consumption with the size of tensors.
While the data in our real world is usually presented as trillion- or even
exascale-scale tensors, existing work can only support billion-scale scale
tensors. In our work, we propose the Exascale-Tensor to mitigate the
significant gap. Specifically, we propose a compression-based tensor
decomposition framework, namely the exascale-tensor, to support exascale tensor
decomposition. Then, we carefully analyze the inherent parallelism and propose
a bag of strategies to improve computational efficiency. Last, we conduct
experiments to decompose tensors ranging from million-scale to trillion-scale
for evaluation. Compared to the baselines, the exascale-tensor supports 8,000x
larger tensors and a speedup up to 6.95x. We also apply our method to two
real-world applications, including gene analysis and tensor layer neural
networks, of which the numeric results demonstrate the scalability and
effectiveness of our method.