A personalized Uncertainty Quantification framework for patient survival
  models: estimating individual uncertainty of patients with metastatic brain
  tumors in the absence of ground truth
Yuqi Wang Aarzu Gupta David Carpenter Trey Mullikin Zachary J. Reitman Scott Floyd John Kirkpatrick Joseph K. Salama Paul W. Sperduto Jian-Guo Liu Mustafa R. Bashir Kyle J. Lafata
TodevelopanovelUncertaintyQuantification (UQ) framework to estimate the
uncertainty of patient survival models in the absence of ground truth, we
developed and evaluated our approach based on a dataset of 1383 patients
treated with stereotactic radiosurgery (SRS) for brain metastases between
January 2015 and December 2020. Our motivating hypothesis is that a
time-to-event prediction of a test patient on inference is more certain given a
higher feature-space-similarity to patients in the training set. Therefore, the
uncertainty for a particular patient-of-interest is represented by the
concordance index between a patient similarity rank and a prediction similarity
rank. Model uncertainty was defined as the increased percentage of the max
uncertainty-constrained-AUC compared to the model AUC. We evaluated our method
on multiple clinically-relevant endpoints, including time to intracranial
progression (ICP), progression-free survival (PFS) after SRS, overall survival
(OS), and time to ICP and/or death (ICPD), on a variety of both statistical and
non-statistical models, including CoxPH, conditional survival forest (CSF), and
neural multi-task linear regression (NMTLR). Our results show that all models
had the lowest uncertainty on ICP (2.21%) and the highest uncertainty (17.28%)
on ICPD. OS models demonstrated high variation in uncertainty performance,
where NMTLR had the lowest uncertainty(1.96%)and CSF had the highest
uncertainty (14.29%). In conclusion, our method can estimate the uncertainty of
individual patient survival modeling results. As expected, our data empirically
demonstrate that as model uncertainty measured via our technique increases, the
similarity between a feature-space and its predicted outcome decreases.