Diagnostic Spatio-temporal Transformer with Faithful Encoding
Jokin Labaien Tsuyoshi Idé Pin-Yu Chen Ekhi Zugasti Xabier De Carlos
This paper addresses the task of anomaly diagnosis when the underlying data
generation process has a complex spatio-temporal (ST) dependency. The key
technical challenge is to extract actionable insights from the dependency
tensor characterizing high-order interactions among temporal and spatial
indices. We formalize the problem as supervised dependency discovery, where the
ST dependency is learned as a side product of multivariate time-series
classification. We show that temporal positional encoding used in existing ST
transformer works has a serious limitation in capturing higher frequencies
(short time scales). We propose a new positional encoding with a theoretical
guarantee, based on discrete Fourier transform. We also propose a new ST
dependency discovery framework, which can provide readily consumable diagnostic
information in both spatial and temporal directions. Finally, we demonstrate
the utility of the proposed model, DFStrans (Diagnostic Fourier-based
Spatio-temporal Transformer), in a real industrial application of building
elevator control.