Information Modified K-Nearest Neighbor
Mohammad Ali Vahedifar Azim Akhtarshenas Mariam Sabbaghian Mohammad Rafatpanah
In this research paper, we introduce a novel classification method aimed at
improving the performance of the K-Nearest Neighbors (KNN) algorithm. Our
approach leverages Mutual Information (MI) to enhance the significance of
weights and draw inspiration from Shapley values, a concept originating from
cooperative game theory, to refine value allocation. The fundamental concept
underlying KNN is the classification of samples based on the majority thorough
their k-nearest neighbors. While both the distances and labels of these
neighbors are crucial, traditional KNN assigns equal weight to all samples and
prevance considering the varying importance of each neighbor based on their
distances and labels.
  In the proposed method, known as Information-Modified KNN (IMKNN), we address
this issue by introducing a straightforward algorithm. To evaluate the
effectiveness of our approach, it is compared with 7 contemporary variants of
KNN, as well as the traditional KNN. Each of these variants exhibits its unique
advantages and limitations. We conduct experiments on 12 widely-used datasets,
assessing the methods' performance in terms of accuracy, precision and recall.
  Our study demonstrates that IMKNN consistently outperforms other methods
across different datasets and criteria by highlighting its superior performance
in various classification tasks. These findings underscore the potential of
IMKNN as a valuable tool for enhancing the capabilities of the KNN algorithm in
diverse applications.