Temporal Smoothness Regularisers for Neural Link Predictors
Manuel Dileo Pasquale Minervini Matteo Zignani Sabrina Gaito
Most algorithms for representation learning and link prediction on relational
data are designed for static data. However, the data to which they are applied
typically evolves over time, including online social networks or interactions
between users and items in recommender systems. This is also the case for
graph-structured knowledge bases -- knowledge graphs -- which contain facts
that are valid only for specific points in time. In such contexts, it becomes
crucial to correctly identify missing links at a precise time point, i.e. the
temporal prediction link task. Recently, Lacroix et al. and Sadeghian et al.
proposed a solution to the problem of link prediction for knowledge graphs
under temporal constraints inspired by the canonical decomposition of 4-order
tensors, where they regularise the representations of time steps by enforcing
temporal smoothing, i.e. by learning similar transformation for adjacent
timestamps. However, the impact of the choice of temporal regularisation terms
is still poorly understood. In this work, we systematically analyse several
choices of temporal smoothing regularisers using linear functions and recurrent
architectures. In our experiments, we show that by carefully selecting the
temporal smoothing regulariser and regularisation weight, a simple method like
TNTComplEx can produce significantly more accurate results than
state-of-the-art methods on three widely used temporal link prediction
datasets. Furthermore, we evaluate the impact of a wide range of temporal
smoothing regularisers on two state-of-the-art temporal link prediction models.
Our work shows that simple tensor factorisation models can produce new
state-of-the-art results using newly proposed temporal regularisers,
highlighting a promising avenue for future research.