The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive
  field
Kun Wang Guohao Li Shilong Wang Guibin Zhang Kai Wang Yang You Xiaojiang Peng Yuxuan Liang Yang Wang
Despite Graph Neural Networks demonstrating considerable promise in graph
representation learning tasks, GNNs predominantly face significant issues with
over-fitting and over-smoothing as they go deeper as models of computer vision
realm. In this work, we conduct a systematic study of deeper GNN research
trajectories. Our findings indicate that the current success of deep GNNs
primarily stems from (I) the adoption of innovations from CNNs, such as
residual/skip connections, or (II) the tailor-made aggregation algorithms like
DropEdge. However, these algorithms often lack intrinsic interpretability and
indiscriminately treat all nodes within a given layer in a similar manner,
thereby failing to capture the nuanced differences among various nodes. To this
end, we introduce the Snowflake Hypothesis -- a novel paradigm underpinning the
concept of ``one node, one receptive field''. The hypothesis draws inspiration
from the unique and individualistic patterns of each snowflake, proposing a
corresponding uniqueness in the receptive fields of nodes in the GNNs.
  We employ the simplest gradient and node-level cosine distance as guiding
principles to regulate the aggregation depth for each node, and conduct
comprehensive experiments including: (1) different training schemes; (2)
various shallow and deep GNN backbones, and (3) various numbers of layers (8,
16, 32, 64) on multiple benchmarks (six graphs including dense graphs with
millions of nodes); (4) compare with different aggregation strategies. The
observational results demonstrate that our hypothesis can serve as a universal
operator for a range of tasks, and it displays tremendous potential on deep
GNNs. It can be applied to various GNN frameworks, enhancing its effectiveness
when operating in-depth, and guiding the selection of the optimal network depth
in an explainable and generalizable way.