What is Hiding in Medicine's Dark Matter? Learning with Missing Data in
  Medical Practices
Neslihan Suzen Evgeny M. Mirkes Damian Roland Jeremy Levesley Alexander N. Gorban Tim J. Coats
Electronic patient records (EPRs) produce a wealth of data but contain
significant missing information. Understanding and handling this missing data
is an important part of clinical data analysis and if left unaddressed could
result in bias in analysis and distortion in critical conclusions. Missing data
may be linked to health care professional practice patterns and imputation of
missing data can increase the validity of clinical decisions. This study
focuses on statistical approaches for understanding and interpreting the
missing data and machine learning based clinical data imputation using a single
centre's paediatric emergency data and the data from UK's largest clinical
audit for traumatic injury database (TARN). In the study of 56,961 data points
related to initial vital signs and observations taken on children presenting to
an Emergency Department, we have shown that missing data are likely to be
non-random and how these are linked to health care professional practice
patterns. We have then examined 79 TARN fields with missing values for 5,791
trauma cases. Singular Value Decomposition (SVD) and k-Nearest Neighbour (kNN)
based missing data imputation methods are used and imputation results against
the original dataset are compared and statistically tested. We have concluded
that the 1NN imputer is the best imputation which indicates a usual pattern of
clinical decision making: find the most similar patients and take their
attributes as imputation.