SECNN: Squeeze-and-Excitation Convolutional Neural Network for Sentence
  Classification
Shandong Yuan
Sentence classification is one of the basic tasks of natural language
processing. Convolution neural network (CNN) has the ability to extract n-grams
features through convolutional filters and capture local correlations between
consecutive words in parallel, so CNN is a popular neural network architecture
to dealing with the task. But restricted by the width of convolutional filters,
it is difficult for CNN to capture long term contextual dependencies. Attention
is a mechanism that considers global information and pays more attention to
keywords in sentences, thus attention mechanism is cooperated with CNN network
to improve performance in sentence classification task. In our work, we don't
focus on keyword in a sentence, but on which CNN's output feature map is more
important. We propose a Squeeze-and-Excitation Convolutional neural Network
(SECNN) for sentence classification. SECNN takes the feature maps from multiple
CNN as different channels of sentence representation, and then, we can utilize
channel attention mechanism, that is SE attention mechanism, to enable the
model to learn the attention weights of different channel features. The results
show that our model achieves advanced performance in the sentence
classification task.