Classifying Whole Slide Images: What Matters?
Long Nguyen Aiden Nibali Joshua Millward Zhen He
Recently there have been many algorithms proposed for the classification of
very high resolution whole slide images (WSIs). These new algorithms are mostly
focused on finding novel ways to combine the information from small local
patches extracted from the slide, with an emphasis on effectively aggregating
more global information for the final predictor. In this paper we thoroughly
explore different key design choices for WSI classification algorithms to
investigate what matters most for achieving high accuracy. Surprisingly, we
found that capturing global context information does not necessarily mean
better performance. A model that captures the most global information
consistently performs worse than a model that captures less global information.
In addition, a very simple multi-instance learning method that captures no
global information performs almost as well as models that capture a lot of
global information. These results suggest that the most important features for
effective WSI classification are captured at the local small patch level, where
cell and tissue micro-environment detail is most pronounced. Another surprising
finding was that unsupervised pre-training on a larger set of 33 cancers gives
significantly worse performance compared to pre-training on a smaller dataset
of 7 cancers (including the target cancer). We posit that pre-training on a
smaller, more focused dataset allows the feature extractor to make better use
of the limited feature space to better discriminate between subtle differences
in the input patch.