Learning with SASQuaTCh: a Novel Variational Quantum Transformer
  Architecture with Kernel-Based Self-Attention
Ethan N. Evans Matthew Cook Zachary P. Bradshaw Margarite L. LaBorde
The widely popular transformer network popularized by the generative
pre-trained transformer (GPT) has a large field of applicability, including
predicting text and images, classification, and even predicting solutions to
the dynamics of physical systems. In the latter context, the continuous analog
of the self-attention mechanism at the heart of transformer networks has been
applied to learning the solutions of partial differential equations and reveals
a convolution kernel nature that can be exploited by the Fourier transform. It
is well known that many quantum algorithms that have provably demonstrated a
speedup over classical algorithms utilize the quantum Fourier transform. In
this work, we explore quantum circuits that can efficiently express a
self-attention mechanism through the perspective of kernel-based operator
learning. In this perspective, we are able to represent deep layers of a vision
transformer network using simple gate operations and a set of multi-dimensional
quantum Fourier transforms. We analyze the computational and parameter
complexity of our novel variational quantum circuit, which we call
Self-Attention Sequential Quantum Transformer Channel (SASQuaTCh), and
demonstrate its utility on simplified classification problems.