Dynamic Decision Tree Ensembles for Energy-Efficient Inference on IoT
  Edge Nodes
Francesco Daghero Alessio Burrello Enrico Macii Paolo Montuschi Massimo Poncino Daniele Jahier Pagliari
With the increasing popularity of Internet of Things (IoT) devices, there is
a growing need for energy-efficient Machine Learning (ML) models that can run
on constrained edge nodes. Decision tree ensembles, such as Random Forests
(RFs) and Gradient Boosting (GBTs), are particularly suited for this task,
given their relatively low complexity compared to other alternatives. However,
their inference time and energy costs are still significant for edge hardware.
Given that said costs grow linearly with the ensemble size, this paper proposes
the use of dynamic ensembles, that adjust the number of executed trees based
both on a latency/energy target and on the complexity of the processed input,
to trade-off computational cost and accuracy. We focus on deploying these
algorithms on multi-core low-power IoT devices, designing a tool that
automatically converts a Python ensemble into optimized C code, and exploring
several optimizations that account for the available parallelism and memory
hierarchy. We extensively benchmark both static and dynamic RFs and GBTs on
three state-of-the-art IoT-relevant datasets, using an 8-core ultra-lowpower
System-on-Chip (SoC), GAP8, as the target platform. Thanks to the proposed
early-stopping mechanisms, we achieve an energy reduction of up to 37.9% with
respect to static GBTs (8.82 uJ vs 14.20 uJ per inference) and 41.7% with
respect to static RFs (2.86 uJ vs 4.90 uJ per inference), without losing
accuracy compared to the static model.