A Transition System Abstraction Framework for Neural Network Dynamical
  System Models
Yejiang Yang Zihao Mo Hoang-Dung Tran Weiming Xiang
This paper proposes a transition system abstraction framework for neural
network dynamical system models to enhance the model interpretability, with
applications to complex dynamical systems such as human behavior learning and
verification. To begin with, the localized working zone will be segmented into
multiple localized partitions under the data-driven Maximum Entropy (ME)
partitioning method. Then, the transition matrix will be obtained based on the
set-valued reachability analysis of neural networks. Finally, applications to
human handwriting dynamics learning and verification are given to validate our
proposed abstraction framework, which demonstrates the advantages of enhancing
the interpretability of the black-box model, i.e., our proposed framework is
able to abstract a data-driven neural network model into a transition system,
making the neural network model interpretable through verifying specifications
described in Computational Tree Logic (CTL) languages.