Unsupervised Discovery of Clinical Disease Signatures Using
  Probabilistic Independence
Thomas A. Lasko John M. Still Thomas Z. Li Marco Barbero Mota William W. Stead Eric V. Strobl Bennett A. Landman Fabien Maldonado
Insufficiently precise diagnosis of clinical disease is likely responsible
for many treatment failures, even for common conditions and treatments. With a
large enough dataset, it may be possible to use unsupervised machine learning
to define clinical disease patterns more precisely. We present an approach to
learning these patterns by using probabilistic independence to disentangle the
imprint on the medical record of causal latent sources of disease. We inferred
a broad set of 2000 clinical signatures of latent sources from 9195 variables
in 269,099 Electronic Health Records. The learned signatures produced better
discrimination than the original variables in a lung cancer prediction task
unknown to the inference algorithm, predicting 3-year malignancy in patients
with no history of cancer before a solitary lung nodule was discovered. More
importantly, the signatures' greater explanatory power identified pre-nodule
signatures of apparently undiagnosed cancer in many of those patients.