Iterative missing value imputation based on feature importance
Cong Guo Chun Liu Wei Yang
Many datasets suffer from missing values due to various reasons,which not
only increases the processing difficulty of related tasks but also reduces the
accuracy of classification. To address this problem, the mainstream approach is
to use missing value imputation to complete the dataset. Existing imputation
methods estimate the missing parts based on the observed values in the original
feature space, and they treat all features as equally important during data
completion, while in fact different features have different importance.
Therefore, we have designed an imputation method that considers feature
importance. This algorithm iteratively performs matrix completion and feature
importance learning, and specifically, matrix completion is based on a filling
loss that incorporates feature importance. Our experimental analysis involves
three types of datasets: synthetic datasets with different noisy features and
missing values, real-world datasets with artificially generated missing values,
and real-world datasets originally containing missing values. The results on
these datasets consistently show that the proposed method outperforms the
existing five imputation algorithms.To the best of our knowledge, this is the
first work that considers feature importance in the imputation model.