Attention De-sparsification Matters: Inducing Diversity in Digital
  Pathology Representation Learning
Saarthak Kapse Srijan Das Jingwei Zhang Rajarsi R. Gupta Joel Saltz Dimitris Samaras Prateek Prasanna
We propose DiRL, a Diversity-inducing Representation Learning technique for
histopathology imaging. Self-supervised learning techniques, such as
contrastive and non-contrastive approaches, have been shown to learn rich and
effective representations of digitized tissue samples with limited pathologist
supervision. Our analysis of vanilla SSL-pretrained models' attention
distribution reveals an insightful observation: sparsity in attention, i.e,
models tends to localize most of their attention to some prominent patterns in
the image. Although attention sparsity can be beneficial in natural images due
to these prominent patterns being the object of interest itself, this can be
sub-optimal in digital pathology; this is because, unlike natural images,
digital pathology scans are not object-centric, but rather a complex phenotype
of various spatially intermixed biological components. Inadequate
diversification of attention in these complex images could result in crucial
information loss. To address this, we leverage cell segmentation to densely
extract multiple histopathology-specific representations, and then propose a
prior-guided dense pretext task for SSL, designed to match the multiple
corresponding representations between the views. Through this, the model learns
to attend to various components more closely and evenly, thus inducing adequate
diversification in attention for capturing context rich representations.
Through quantitative and qualitative analysis on multiple tasks across cancer
types, we demonstrate the efficacy of our method and observe that the attention
is more globally distributed.