Structure-Aware DropEdge Towards Deep Graph Convolutional Networks
Jiaqi Han Wenbing Huang Yu Rong Tingyang Xu Fuchun Sun Junzhou Huang
It has been discovered that Graph Convolutional Networks (GCNs) encounter a
remarkable drop in performance when multiple layers are piled up. The main
factor that accounts for why deep GCNs fail lies in over-smoothing, which
isolates the network output from the input with the increase of network depth,
weakening expressivity and trainability. In this paper, we start by
investigating refined measures upon DropEdge -- an existing simple yet
effective technique to relieve over-smoothing. We term our method as DropEdge++
for its two structure-aware samplers in contrast to DropEdge: layer-dependent
sampler and feature-dependent sampler. Regarding the layer-dependent sampler,
we interestingly find that increasingly sampling edges from the bottom layer
yields superior performance than the decreasing counterpart as well as
DropEdge. We theoretically reveal this phenomenon with Mean-Edge-Number (MEN),
a metric closely related to over-smoothing. For the feature-dependent sampler,
we associate the edge sampling probability with the feature similarity of node
pairs, and prove that it further correlates the convergence subspace of the
output layer with the input features. Extensive experiments on several node
classification benchmarks, including both full- and semi- supervised tasks,
illustrate the efficacy of DropEdge++ and its compatibility with a variety of
backbones by achieving generally better performance over DropEdge and the
no-drop version.