Semantically Redundant Training Data Removal and Deep Model
  Classification Performance: A Study with Chest X-rays
Sivaramakrishnan Rajaraman Ghada Zamzmi Feng Yang Zhaohui Liang Zhiyun Xue Sameer Antani
Deep learning (DL) has demonstrated its innate capacity to independently
learn hierarchical features from complex and multi-dimensional data. A common
understanding is that its performance scales up with the amount of training
data. Another data attribute is the inherent variety. It follows, therefore,
that semantic redundancy, which is the presence of similar or repetitive
information, would tend to lower performance and limit generalizability to
unseen data. In medical imaging data, semantic redundancy can occur due to the
presence of multiple images that have highly similar presentations for the
disease of interest. Further, the common use of augmentation methods to
generate variety in DL training may be limiting performance when applied to
semantically redundant data. We propose an entropy-based sample scoring
approach to identify and remove semantically redundant training data. We
demonstrate using the publicly available NIH chest X-ray dataset that the model
trained on the resulting informative subset of training data significantly
outperforms the model trained on the full training set, during both internal
(recall: 0.7164 vs 0.6597, p<0.05) and external testing (recall: 0.3185 vs
0.2589, p<0.05). Our findings emphasize the importance of information-oriented
training sample selection as opposed to the conventional practice of using all
available training data.