MA-VAE: Multi-head Attention-based Variational Autoencoder Approach for
  Anomaly Detection in Multivariate Time-series Applied to Automotive Endurance
  Powertrain Testing
Lucas Correia Jan-Christoph Goos Philipp Klein Thomas Bäck Anna V. Kononova
A clear need for automatic anomaly detection applied to automotive testing
has emerged as more and more attention is paid to the data recorded and manual
evaluation by humans reaches its capacity. Such real-world data is massive,
diverse, multivariate and temporal in nature, therefore requiring modelling of
the testee behaviour. We propose a variational autoencoder with multi-head
attention (MA-VAE), which, when trained on unlabelled data, not only provides
very few false positives but also manages to detect the majority of the
anomalies presented. In addition to that, the approach offers a novel way to
avoid the bypass phenomenon, an undesirable behaviour investigated in
literature. Lastly, the approach also introduces a new method to remap
individual windows to a continuous time series. The results are presented in
the context of a real-world industrial data set and several experiments are
undertaken to further investigate certain aspects of the proposed model. When
configured properly, it is 9% of the time wrong when an anomaly is flagged and
discovers 67% of the anomalies present. Also, MA-VAE has the potential to
perform well with only a fraction of the training and validation subset,
however, to extract it, a more sophisticated threshold estimation method is
required.