MobileViG: Graph-Based Sparse Attention for Mobile Vision Applications
Mustafa Munir William Avery Radu Marculescu
Traditionally, convolutional neural networks (CNN) and vision transformers
(ViT) have dominated computer vision. However, recently proposed vision graph
neural networks (ViG) provide a new avenue for exploration. Unfortunately, for
mobile applications, ViGs are computationally expensive due to the overhead of
representing images as graph structures. In this work, we propose a new
graph-based sparse attention mechanism, Sparse Vision Graph Attention (SVGA),
that is designed for ViGs running on mobile devices. Additionally, we propose
the first hybrid CNN-GNN architecture for vision tasks on mobile devices,
MobileViG, which uses SVGA. Extensive experiments show that MobileViG beats
existing ViG models and existing mobile CNN and ViT architectures in terms of
accuracy and/or speed on image classification, object detection, and instance
segmentation tasks. Our fastest model, MobileViG-Ti, achieves 75.7% top-1
accuracy on ImageNet-1K with 0.78 ms inference latency on iPhone 13 Mini NPU
(compiled with CoreML), which is faster than MobileNetV2x1.4 (1.02 ms, 74.7%
top-1) and MobileNetV2x1.0 (0.81 ms, 71.8% top-1). Our largest model,
MobileViG-B obtains 82.6% top-1 accuracy with only 2.30 ms latency, which is
faster and more accurate than the similarly sized EfficientFormer-L3 model
(2.77 ms, 82.4%). Our work proves that well designed hybrid CNN-GNN
architectures can be a new avenue of exploration for designing models that are
extremely fast and accurate on mobile devices. Our code is publicly available
at https://github.com/SLDGroup/MobileViG.