Concept Distillation: Leveraging Human-Centered Explanations for Model
  Improvement
Avani Gupta Saurabh Saini P J Narayanan
Humans use abstract concepts for understanding instead of hard features.
Recent interpretability research has focused on human-centered concept
explanations of neural networks. Concept Activation Vectors (CAVs) estimate a
model's sensitivity and possible biases to a given concept. In this paper, we
extend CAVs from post-hoc analysis to ante-hoc training in order to reduce
model bias through fine-tuning using an additional Concept Loss. Concepts were
defined on the final layer of the network in the past. We generalize it to
intermediate layers using class prototypes. This facilitates class learning in
the last convolution layer, which is known to be most informative. We also
introduce Concept Distillation to create richer concepts using a pre-trained
knowledgeable model as the teacher. Our method can sensitize or desensitize a
model towards concepts. We show applications of concept-sensitive training to
debias several classification problems. We also use concepts to induce prior
knowledge into IID, a reconstruction problem. Concept-sensitive training can
improve model interpretability, reduce biases, and induce prior knowledge.
Please visit https://avani17101.github.io/Concept-Distilllation/ for code and
more details.