Describe-and-Dissect: Interpreting Neurons in Vision Networks with
  Language Models
Nicholas Bai Rahul A. Iyer Tuomas Oikarinen Tsui-Wei Weng
In this paper, we propose Describe-and-Dissect (DnD), a novel method to
describe the roles of hidden neurons in vision networks. DnD utilizes recent
advancements in multimodal deep learning to produce complex natural language
descriptions, without the need for labeled training data or a predefined set of
concepts to choose from. Additionally, DnD is training-free, meaning we don't
train any new models and can easily leverage more capable general purpose
models in the future. We have conducted extensive qualitative and quantitative
analysis to show that DnD outperforms prior work by providing higher quality
neuron descriptions. Specifically, our method on average provides the highest
quality labels and is more than 2 times as likely to be selected as the best
explanation for a neuron than the best baseline.