Pretext Training Algorithms for Event Sequence Data
Yimu Wang He Zhao Ruizhi Deng Frederick Tung Greg Mori
Pretext training followed by task-specific fine-tuning has been a successful
approach in vision and language domains. This paper proposes a self-supervised
pretext training framework tailored to event sequence data. We introduce a
novel alignment verification task that is specialized to event sequences,
building on good practices in masked reconstruction and contrastive learning.
Our pretext tasks unlock foundational representations that are generalizable
across different down-stream tasks, including next-event prediction for
temporal point process models, event sequence classification, and missing event
interpolation. Experiments on popular public benchmarks demonstrate the
potential of the proposed method across different tasks and data domains.