FeaInfNet: Diagnosis in Medical Image with Feature-Driven Inference and
  Visual Explanations
Yitao Peng Lianghua He Die Hu Yihang Liu Longzhen Yang Shaohua Shang
Interpretable deep learning models have received widespread attention in the
field of image recognition. Due to the unique multi-instance learning of
medical images and the difficulty in identifying decision-making regions, many
interpretability models that have been proposed still have problems of
insufficient accuracy and interpretability in medical image disease diagnosis.
To solve these problems, we propose feature-driven inference network
(FeaInfNet). Our first key innovation involves proposing a feature-based
network reasoning structure, which is applied to FeaInfNet. The network of this
structure compares the similarity of each sub-region image patch with the
disease templates and normal templates that may appear in the region, and
finally combines the comparison of each sub-region to make the final diagnosis.
It simulates the diagnosis process of doctors to make the model interpretable
in the reasoning process, while avoiding the misleading caused by the
participation of normal areas in reasoning. Secondly, we propose local feature
masks (LFM) to extract feature vectors in order to provide global information
for these vectors, thus enhancing the expressive ability of the FeaInfNet.
Finally, we propose adaptive dynamic masks (Adaptive-DM) to interpret feature
vectors and prototypes into human-understandable image patches to provide
accurate visual interpretation. We conducted qualitative and quantitative
experiments on multiple publicly available medical datasets, including RSNA,
iChallenge-PM, Covid-19, ChinaCXRSet, and MontgomerySet. The results of our
experiments validate that our method achieves state-of-the-art performance in
terms of classification accuracy and interpretability compared to baseline
methods in medical image diagnosis. Additional ablation studies verify the
effectiveness of each of our proposed components.