Predicting Ovarian Cancer Treatment Response in Histopathology using
  Hierarchical Vision Transformers and Multiple Instance Learning
Jack Breen Katie Allen Kieran Zucker Geoff Hall Nishant Ravikumar Nicolas M. Orsi
For many patients, current ovarian cancer treatments offer limited clinical
benefit. For some therapies, it is not possible to predict patients' responses,
potentially exposing them to the adverse effects of treatment without any
therapeutic benefit. As part of the automated prediction of treatment
effectiveness in ovarian cancer using histopathological images (ATEC23)
challenge, we evaluated the effectiveness of deep learning to predict whether a
course of treatment including the antiangiogenic drug bevacizumab could
contribute to remission or prevent disease progression for at least 6 months in
a set of 282 histopathology whole slide images (WSIs) from 78 ovarian cancer
patients. Our approach used a pretrained Hierarchical Image Pyramid Transformer
(HIPT) to extract region-level features and an attention-based multiple
instance learning (ABMIL) model to aggregate features and classify whole
slides. The optimal HIPT-ABMIL model had an internal balanced accuracy of 60.2%
+- 2.9% and an AUC of 0.646 +- 0.033. Histopathology-specific model pretraining
was found to be beneficial to classification performance, though hierarchical
transformers were not, with a ResNet feature extractor achieving similar
performance. Due to the dataset being small and highly heterogeneous,
performance was variable across 5-fold cross-validation folds, and there were
some extreme differences between validation and test set performance within
folds. The model did not generalise well to tissue microarrays, with accuracy
worse than random chance. It is not yet clear whether ovarian cancer WSIs
contain information that can be used to accurately predict treatment response,
with further validation using larger, higher-quality datasets required.