Learning Saliency From Fixations
Yasser Abdelaziz Dahou Djilali Kevin McGuiness Noel O'Connor
We present a novel approach for saliency prediction in images, leveraging
parallel decoding in transformers to learn saliency solely from fixation maps.
Models typically rely on continuous saliency maps, to overcome the difficulty
of optimizing for the discrete fixation map. We attempt to replicate the
experimental setup that generates saliency datasets. Our approach treats
saliency prediction as a direct set prediction problem, via a global loss that
enforces unique fixations prediction through bipartite matching and a
transformer encoder-decoder architecture. By utilizing a fixed set of learned
fixation queries, the cross-attention reasons over the image features to
directly output the fixation points, distinguishing it from other modern
saliency predictors. Our approach, named Saliency TRansformer (SalTR), achieves
metric scores on par with state-of-the-art approaches on the Salicon and MIT300
benchmarks.