On the Concept Trustworthiness in Concept Bottleneck Models
Qihan Huang Jie Song Jingwen Hu Haofei Zhang Yong Wang Mingli Song
Concept Bottleneck Models (CBMs), which break down the reasoning process into
the input-to-concept mapping and the concept-to-label prediction, have garnered
significant attention due to their remarkable interpretability achieved by the
interpretable concept bottleneck. However, despite the transparency of the
concept-to-label prediction, the mapping from the input to the intermediate
concept remains a black box, giving rise to concerns about the trustworthiness
of the learned concepts (i.e., these concepts may be predicted based on
spurious cues). The issue of concept untrustworthiness greatly hampers the
interpretability of CBMs, thereby hindering their further advancement. To
conduct a comprehensive analysis on this issue, in this study we establish a
benchmark to assess the trustworthiness of concepts in CBMs. A pioneering
metric, referred to as concept trustworthiness score, is proposed to gauge
whether the concepts are derived from relevant regions. Additionally, an
enhanced CBM is introduced, enabling concept predictions to be made
specifically from distinct parts of the feature map, thereby facilitating the
exploration of their related regions. Besides, we introduce three modules,
namely the cross-layer alignment (CLA) module, the cross-image alignment (CIA)
module, and the prediction alignment (PA) module, to further enhance the
concept trustworthiness within the elaborated CBM. The experiments on five
datasets across ten architectures demonstrate that without using any concept
localization annotations during training, our model improves the concept
trustworthiness by a large margin, meanwhile achieving superior accuracy to the
state-of-the-arts. Our code is available at https://github.com/hqhQAQ/ProtoCBM.