Riemannian Self-Attention Mechanism for SPD Networks
Rui Wang Xiao-Jun Wu Hui Li Josef Kittler
Symmetric positive definite (SPD) matrix has been demonstrated to be an
effective feature descriptor in many scientific areas, as it can encode
spatiotemporal statistics of the data adequately on a curved Riemannian
manifold, i.e., SPD manifold. Although there are many different ways to design
network architectures for SPD matrix nonlinear learning, very few solutions
explicitly mine the geometrical dependencies of features at different layers.
Motivated by the great success of self-attention mechanism in capturing
long-range relationships, an SPD manifold self-attention mechanism (SMSA) is
proposed in this paper using some manifold-valued geometric operations, mainly
the Riemannian metric, Riemannian mean, and Riemannian optimization. Then, an
SMSA-based geometric learning module (SMSA-GLM) is designed for the sake of
improving the discrimination of the generated deep structured representations.
Extensive experimental results achieved on three benchmarking datasets show
that our modification against the baseline network further alleviates the
information degradation problem and leads to improved accuracy.