MAMI: Multi-Attentional Mutual-Information for Long Sequence Neuron
  Captioning
Alfirsa Damasyifa Fauzulhaq Wahyu Parwitayasa Joseph Ananda Sugihdharma M. Fadli Ridhani Novanto Yudistira
Neuron labeling is an approach to visualize the behaviour and respond of a
certain neuron to a certain pattern that activates the neuron. Neuron labeling
extract information about the features captured by certain neurons in a deep
neural network, one of which uses the encoder-decoder image captioning
approach. The encoder used can be a pretrained CNN-based model and the decoder
is an RNN-based model for text generation. Previous work, namely MILAN (Mutual
Information-guided Linguistic Annotation of Neuron), has tried to visualize the
neuron behaviour using modified Show, Attend, and Tell (SAT) model in the
encoder, and LSTM added with Bahdanau attention in the decoder. MILAN can show
great result on short sequence neuron captioning, but it does not show great
result on long sequence neuron captioning, so in this work, we would like to
improve the performance of MILAN even more by utilizing different kind of
attention mechanism and additionally adding several attention result into one,
in order to combine all the advantages from several attention mechanism. Using
our compound dataset, we obtained higher BLEU and F1-Score on our proposed
model, achieving 17.742 and 0.4811 respectively. At some point where the model
converges at the peak, our model obtained BLEU of 21.2262 and BERTScore
F1-Score of 0.4870.