Edge Devices Inference Performance Comparison
R. Tobiasz G. Wilczyński P. Graszka N. Czechowski S. Łuczak
In this work, we investigate the inference time of the MobileNet family,
EfficientNet V1 and V2 family, VGG models, Resnet family, and InceptionV3 on
four edge platforms. Specifically NVIDIA Jetson Nano, Intel Neural Stick,
Google Coral USB Dongle, and Google Coral PCIe. Our main contribution is a
thorough analysis of the aforementioned models in multiple settings, especially
as a function of input size, the presence of the classification head, its size,
and the scale of the model. Since throughout the industry, those architectures
are mainly utilized as feature extractors we put our main focus on analyzing
them as such. We show that Google platforms offer the fastest average inference
time, especially for newer models like MobileNet or EfficientNet family, while
Intel Neural Stick is the most universal accelerator allowing to run most
architectures. These results should provide guidance for engineers in the early
stages of AI edge systems development. All of them are accessible at
https://bulletprove.com/research/edge_inference_results.csv