Spatiotemporal Attention-based Semantic Compression for Real-time Video
  Recognition
Nan Li Mehdi Bennis Alexandros Iosifidis Qi Zhang
This paper studies the computational offloading of video action recognition
in edge computing. To achieve effective semantic information extraction and
compression, following semantic communication we propose a novel spatiotemporal
attention-based autoencoder (STAE) architecture, including a frame attention
module and a spatial attention module, to evaluate the importance of frames and
pixels in each frame. Additionally, we use entropy encoding to remove
statistical redundancy in the compressed data to further reduce communication
overhead. At the receiver, we develop a lightweight decoder that leverages a
3D-2D CNN combined architecture to reconstruct missing information by
simultaneously learning temporal and spatial information from the received data
to improve accuracy. To fasten convergence, we use a step-by-step approach to
train the resulting STAE-based vision transformer (ViT_STAE) models.
Experimental results show that ViT_STAE can compress the video dataset HMDB51
by 104x with only 5% accuracy loss, outperforming the state-of-the-art baseline
DeepISC. The proposed ViT_STAE achieves faster inference and higher accuracy
than the DeepISC-based ViT model under time-varying wireless channel, which
highlights the effectiveness of STAE in guaranteeing higher accuracy under time
constraints.