Extracting Interpretable Local and Global Representations from Attention
  on Time Series
Leonid Schwenke Martin Atzmueller
This paper targets two transformer attention based interpretability methods
working with local abstraction and global representation, in the context of
time series data. We distinguish local and global contexts, and provide a
comprehensive framework for both general interpretation options. We discuss
their specific instantiation via different methods in detail, also outlining
their respective computational implementation and abstraction variants.
Furthermore, we provide extensive experimentation demonstrating the efficacy of
the presented approaches. In particular, we perform our experiments using a
selection of univariate datasets from the UCR UEA time series repository where
we both assess the performance of the proposed approaches, as well as their
impact on explainability and interpretability/complexity. Here, with an
extensive analysis of hyperparameters, the presented approaches demonstrate an
significant improvement in interpretability/complexity, while capturing many
core decisions of and maintaining a similar performance to the baseline model.
Finally, we draw general conclusions outlining and guiding the application of
the presented methods.