Uncertainty Quantification in Detecting Choroidal Metastases on MRI via
  Evolutionary Strategies
Bala McRae-Posani Andrei Holodny Hrithwik Shalu Joseph N Stember
Uncertainty quantification plays a vital role in facilitating the practical
implementation of AI in radiology by addressing growing concerns around
trustworthiness. Given the challenges associated with acquiring large,
annotated datasets in this field, there is a need for methods that enable
uncertainty quantification in small data AI approaches tailored to radiology
images. In this study, we focused on uncertainty quantification within the
context of the small data evolutionary strategies-based technique of deep
neuroevolution (DNE). Specifically, we employed DNE to train a simple
Convolutional Neural Network (CNN) with MRI images of the eyes for binary
classification. The goal was to distinguish between normal eyes and those with
metastatic tumors called choroidal metastases. The training set comprised 18
images with choroidal metastases and 18 without tumors, while the testing set
contained a tumor-to-normal ratio of 15:15.
  We trained CNN model weights via DNE for approximately 40,000 episodes,
ultimately reaching a convergence of 100% accuracy on the training set. We
saved all models that achieved maximal training set accuracy. Then, by applying
these models to the testing set, we established an ensemble method for
uncertainty quantification.The saved set of models produced distributions for
each testing set image between the two classes of normal and tumor-containing.
The relative frequencies permitted uncertainty quantification of model
predictions. Intriguingly, we found that subjective features appreciated by
human radiologists explained images for which uncertainty was high,
highlighting the significance of uncertainty quantification in AI-driven
radiological analyses.