Representation Learning of Multivariate Time Series using Attention and
  Adversarial Training
Leon Scharwächter Sebastian Otte
A critical factor in trustworthy machine learning is to develop robust
representations of the training data. Only under this guarantee methods are
legitimate to artificially generate data, for example, to counteract imbalanced
datasets or provide counterfactual explanations for blackbox decision-making
systems. In recent years, Generative Adversarial Networks (GANs) have shown
considerable results in forming stable representations and generating realistic
data. While many applications focus on generating image data, less effort has
been made in generating time series data, especially multivariate signals. In
this work, a Transformer-based autoencoder is proposed that is regularized
using an adversarial training scheme to generate artificial multivariate time
series signals. The representation is evaluated using t-SNE visualizations,
Dynamic Time Warping (DTW) and Entropy scores. Our results indicate that the
generated signals exhibit higher similarity to an exemplary dataset than using
a convolutional network approach.