Decoding the visual attention of pathologists to reveal their level of
  expertise
Souradeep Chakraborty Dana Perez Paul Friedman Natallia Sheuka Constantin Friedman Oksana Yaskiv Rajarsi Gupta Gregory J. Zelinsky Joel H. Saltz Dimitris Samaras
We present a method for classifying the expertise of a pathologist based on
how they allocated their attention during a cancer reading. We engage this
decoding task by developing a novel method for predicting the attention of
pathologists as they read whole-slide Images (WSIs) of prostate and make cancer
grade classifications. Our ground truth measure of a pathologists' attention is
the x, y and z (magnification) movement of their viewport as they navigated
through WSIs during readings, and to date we have the attention behavior of 43
pathologists reading 123 WSIs. These data revealed that specialists have higher
agreement in both their attention and cancer grades compared to general
pathologists and residents, suggesting that sufficient information may exist in
their attention behavior to classify their expertise level. To attempt this, we
trained a transformer-based model to predict the visual attention heatmaps of
resident, general, and specialist (GU) pathologists during Gleason grading.
Based solely on a pathologist's attention during a reading, our model was able
to predict their level of expertise with 75.3%, 56.1%, and 77.2% accuracy,
respectively, better than chance and baseline models. Our model therefore
enables a pathologist's expertise level to be easily and objectively evaluated,
important for pathology training and competency assessment. Tools developed
from our model could also be used to help pathology trainees learn how to read
WSIs like an expert.