Graph Neural Networks Automated Design and Deployment on Device-Edge
  Co-Inference Systems
Ao Zhou Jianlei Yang Tong Qiao Yingjie Qi Zhi Yang Weisheng Zhao Chunming Hu
The key to device-edge co-inference paradigm is to partition models into
computation-friendly and computation-intensive parts across the device and the
edge, respectively. However, for Graph Neural Networks (GNNs), we find that
simply partitioning without altering their structures can hardly achieve the
full potential of the co-inference paradigm due to various
computational-communication overheads of GNN operations over heterogeneous
devices. We present GCoDE, the first automatic framework for GNN that
innovatively Co-designs the architecture search and the mapping of each
operation on Device-Edge hierarchies. GCoDE abstracts the device communication
process into an explicit operation and fuses the search of architecture and the
operations mapping in a unified space for joint-optimization. Also, the
performance-awareness approach, utilized in the constraint-based search process
of GCoDE, enables effective evaluation of architecture efficiency in diverse
heterogeneous systems. We implement the co-inference engine and runtime
dispatcher in GCoDE to enhance the deployment efficiency. Experimental results
show that GCoDE can achieve up to $44.9\times$ speedup and $98.2\%$ energy
reduction compared to existing approaches across various applications and
system configurations.