Adaptive Workload Distribution for Accuracy-aware DNN Inference on
  Collaborative Edge Platforms
Zain Taufique Antonio Miele Pasi Liljeberg Anil Kanduri
DNN inference can be accelerated by distributing the workload among a cluster
of collaborative edge nodes. Heterogeneity among edge devices and
accuracy-performance trade-offs of DNN models present a complex exploration
space while catering to the inference performance requirements. In this work,
we propose adaptive workload distribution for DNN inference, jointly
considering node-level heterogeneity of edge devices, and application-specific
accuracy and performance requirements. Our proposed approach combinatorially
optimizes heterogeneity-aware workload partitioning and dynamic accuracy
configuration of DNN models to ensure performance and accuracy guarantees. We
tested our approach on an edge cluster of Odroid XU4, Raspberry Pi4, and Jetson
Nano boards and achieved an average gain of 41.52% in performance and 5.2% in
output accuracy as compared to state-of-the-art workload distribution
strategies.