Dynamic Tensor Decomposition via Neural Diffusion-Reaction Processes
Zheng Wang Shikai Fang Shibo Li Shandian Zhe
Tensor decomposition is an important tool for multiway data analysis. In
practice, the data is often sparse yet associated with rich temporal
information. Existing methods, however, often under-use the time information
and ignore the structural knowledge within the sparsely observed tensor
entries. To overcome these limitations and to better capture the underlying
temporal structure, we propose Dynamic EMbedIngs fOr dynamic Tensor
dEcomposition (DEMOTE). We develop a neural diffusion-reaction process to
estimate dynamic embeddings for the entities in each tensor mode. Specifically,
based on the observed tensor entries, we build a multi-partite graph to encode
the correlation between the entities. We construct a graph diffusion process to
co-evolve the embedding trajectories of the correlated entities and use a
neural network to construct a reaction process for each individual entity. In
this way, our model can capture both the commonalities and personalities during
the evolution of the embeddings for different entities. We then use a neural
network to model the entry value as a nonlinear function of the embedding
trajectories. For model estimation, we combine ODE solvers to develop a
stochastic mini-batch learning algorithm. We propose a stratified sampling
method to balance the cost of processing each mini-batch so as to improve the
overall efficiency. We show the advantage of our approach in both simulation
study and real-world applications. The code is available at
https://github.com/wzhut/Dynamic-Tensor-Decomposition-via-Neural-Diffusion-Reaction-Processes.