MCA: Moment Channel Attention Networks
Yangbo Jiang Zhiwei Jiang Le Han Zenan Huang Nenggan Zheng
Channel attention mechanisms endeavor to recalibrate channel weights to
enhance representation abilities of networks. However, mainstream methods often
rely solely on global average pooling as the feature squeezer, which
significantly limits the overall potential of models. In this paper, we
investigate the statistical moments of feature maps within a neural network.
Our findings highlight the critical role of high-order moments in enhancing
model capacity. Consequently, we introduce a flexible and comprehensive
mechanism termed Extensive Moment Aggregation (EMA) to capture the global
spatial context. Building upon this mechanism, we propose the Moment Channel
Attention (MCA) framework, which efficiently incorporates multiple levels of
moment-based information while minimizing additional computation costs through
our Cross Moment Convolution (CMC) module. The CMC module via channel-wise
convolution layer to capture multiple order moment information as well as cross
channel features. The MCA block is designed to be lightweight and easily
integrated into a variety of neural network architectures. Experimental results
on classical image classification, object detection, and instance segmentation
tasks demonstrate that our proposed method achieves state-of-the-art results,
outperforming existing channel attention methods.