Memory-Efficient Prompt Tuning for Incremental Histopathology
  Classification
Yu Zhu Kang Li Lequan Yu Pheng-Ann Heng
Recent studies have made remarkable progress in histopathology
classification. Based on current successes, contemporary works proposed to
further upgrade the model towards a more generalizable and robust direction
through incrementally learning from the sequentially delivered domains. Unlike
previous parameter isolation based approaches that usually demand massive
computation resources during model updating, we present a memory-efficient
prompt tuning framework to cultivate model generalization potential in
economical memory cost. For each incoming domain, we reuse the existing
parameters of the initial classification model and attach lightweight trainable
prompts into it for customized tuning. Considering the domain heterogeneity, we
perform decoupled prompt tuning, where we adopt a domain-specific prompt for
each domain to independently investigate its distinctive characteristics, and
one domain-invariant prompt shared across all domains to continually explore
the common content embedding throughout time. All domain-specific prompts will
be appended to the prompt bank and isolated from further changes to prevent
forgetting the distinctive features of early-seen domains. While the
domain-invariant prompt will be passed on and iteratively evolve by
style-augmented prompt refining to improve model generalization capability over
time. In specific, we construct a graph with existing prompts and build a
style-augmented graph attention network to guide the domain-invariant prompt
exploring the overlapped latent embedding among all delivered domains for more
domain generic representations. We have extensively evaluated our framework
with two histopathology tasks, i.e., breast cancer metastasis classification
and epithelium-stroma tissue classification, where our approach yielded
superior performance and memory efficiency over the competing methods.