GraphCleaner: Detecting Mislabelled Samples in Popular Graph Learning
  Benchmarks
Yuwen Li Miao Xiong Bryan Hooi
Label errors have been found to be prevalent in popular text, vision, and
audio datasets, which heavily influence the safe development and evaluation of
machine learning algorithms. Despite increasing efforts towards improving the
quality of generic data types, such as images and texts, the problem of
mislabel detection in graph data remains underexplored. To bridge the gap, we
explore mislabelling issues in popular real-world graph datasets and propose
GraphCleaner, a post-hoc method to detect and correct these mislabelled nodes
in graph datasets. GraphCleaner combines the novel ideas of 1) Synthetic
Mislabel Dataset Generation, which seeks to generate realistic mislabels; and
2) Neighborhood-Aware Mislabel Detection, where neighborhood dependency is
exploited in both labels and base classifier predictions. Empirical evaluations
on 6 datasets and 6 experimental settings demonstrate that GraphCleaner
outperforms the closest baseline, with an average improvement of 0.14 in F1
score, and 0.16 in MCC. On real-data case studies, GraphCleaner detects real
and previously unknown mislabels in popular graph benchmarks: PubMed, Cora,
CiteSeer and OGB-arxiv; we find that at least 6.91% of PubMed data is
mislabelled or ambiguous, and simply removing these mislabelled data can boost
evaluation performance from 86.71% to 89.11%.