DAS: A Deformable Attention to Capture Salient Information in CNNs
Farzad Salajegheh Nader Asadi Soroush Saryazdi Sudhir Mudur
Convolutional Neural Networks (CNNs) excel in local spatial pattern
recognition. For many vision tasks, such as object recognition and
segmentation, salient information is also present outside CNN's kernel
boundaries. However, CNNs struggle in capturing such relevant information due
to their confined receptive fields. Self-attention can improve a model's access
to global information but increases computational overhead. We present a fast
and simple fully convolutional method called DAS that helps focus attention on
relevant information. It uses deformable convolutions for the location of
pertinent image regions and separable convolutions for efficiency. DAS plugs
into existing CNNs and propagates relevant information using a gating
mechanism. Compared to the O(n^2) computational complexity of transformer-style
attention, DAS is O(n). Our claim is that DAS's ability to pay increased
attention to relevant features results in performance improvements when added
to popular CNNs for Image Classification and Object Detection. For example, DAS
yields an improvement on Stanford Dogs (4.47%), ImageNet (1.91%), and COCO AP
(3.3%) with base ResNet50 backbone. This outperforms other CNN attention
mechanisms while using similar or less FLOPs. Our code will be publicly
available.