Prospector Heads: Generalized Feature Attribution for Large Models &
  Data
Gautam Machiraju Alexander Derry Arjun Desai Neel Guha Amir-Hossein Karimi James Zou Russ Altman Christopher Ré Parag Mallick
Feature attribution, the ability to localize regions of the input data that
are relevant for classification, is an important capability for machine
learning models in scientific and biomedical domains. Current methods for
feature attribution, which rely on "explaining" the predictions of end-to-end
classifiers, suffer from imprecise feature localization and are inadequate for
use with small sample sizes and high-dimensional datasets due to computational
challenges. We introduce prospector heads, an efficient and interpretable
alternative to explanation-based methods for feature attribution that can be
applied to any encoder and any data modality. Prospector heads generalize
across modalities through experiments on sequences (text), images (pathology),
and graphs (protein structures), outperforming baseline attribution methods by
up to 49 points in mean localization AUPRC. We also demonstrate how prospector
heads enable improved interpretation and discovery of class-specific patterns
in the input data. Through their high performance, flexibility, and
generalizability, prospectors provide a framework for improving trust and
transparency for machine learning models in complex domains.