Image Compression using only Attention based Neural Networks
Natacha Luka Romain Negrel David Picard
In recent research, Learned Image Compression has gained prominence for its
capacity to outperform traditional handcrafted pipelines, especially at low
bit-rates. While existing methods incorporate convolutional priors with
occasional attention blocks to address long-range dependencies, recent advances
in computer vision advocate for a transformative shift towards fully
transformer-based architectures grounded in the attention mechanism. This paper
investigates the feasibility of image compression exclusively using attention
layers within our novel model, QPressFormer. We introduce the concept of
learned image queries to aggregate patch information via cross-attention,
followed by quantization and coding techniques. Through extensive evaluations,
our work demonstrates competitive performance achieved by convolution-free
architectures across the popular Kodak, DIV2K, and CLIC datasets.