PrACTiS: Perceiver-Attentional Copulas for Time Series
Cat P. Le Chris Cannella Ali Hasan Yuting Ng Vahid Tarokh
Transformers incorporating copula structures have demonstrated remarkable
performance in time series prediction. However, their heavy reliance on
self-attention mechanisms demands substantial computational resources, thus
limiting their practical utility across a wide range of tasks. In this work, we
present a model that combines the perceiver architecture with a copula
structure to enhance time-series forecasting. By leveraging the perceiver as
the encoder, we efficiently transform complex, high-dimensional, multimodal
data into a compact latent space, thereby significantly reducing computational
demands. To further reduce complexity, we introduce midpoint inference and
local attention mechanisms, enabling the model to capture dependencies within
imputed samples effectively. Subsequently, we deploy the copula-based attention
and output variance testing mechanism to capture the joint distribution of
missing data, while simultaneously mitigating error propagation during
prediction. Our experimental results on the unimodal and multimodal benchmarks
showcase a consistent 20\% improvement over the state-of-the-art methods, while
utilizing less than half of available memory resources.