History Repeats: Overcoming Catastrophic Forgetting For Event-Centric
  Temporal Knowledge Graph Completion
Mehrnoosh Mirtaheri Mohammad Rostami Aram Galstyan
Temporal knowledge graph (TKG) completion models typically rely on having
access to the entire graph during training. However, in real-world scenarios,
TKG data is often received incrementally as events unfold, leading to a dynamic
non-stationary data distribution over time. While one could incorporate
fine-tuning to existing methods to allow them to adapt to evolving TKG data,
this can lead to forgetting previously learned patterns. Alternatively,
retraining the model with the entire updated TKG can mitigate forgetting but is
computationally burdensome. To address these challenges, we propose a general
continual training framework that is applicable to any TKG completion method,
and leverages two key ideas: (i) a temporal regularization that encourages
repurposing of less important model parameters for learning new knowledge, and
(ii) a clustering-based experience replay that reinforces the past knowledge by
selectively preserving only a small portion of the past data. Our experimental
results on widely used event-centric TKG datasets demonstrate the effectiveness
of our proposed continual training framework in adapting to new events while
reducing catastrophic forgetting. Further, we perform ablation studies to show
the effectiveness of each component of our proposed framework. Finally, we
investigate the relation between the memory dedicated to experience replay and
the benefit gained from our clustering-based sampling strategy.