Efficient Neural Networks for Tiny Machine Learning: A Comprehensive
  Review
Minh Tri Lê Pierre Wolinski Julyan Arbel
The field of Tiny Machine Learning (TinyML) has gained significant attention
due to its potential to enable intelligent applications on resource-constrained
devices. This review provides an in-depth analysis of the advancements in
efficient neural networks and the deployment of deep learning models on
ultra-low power microcontrollers (MCUs) for TinyML applications. It begins by
introducing neural networks and discussing their architectures and resource
requirements. It then explores MEMS-based applications on ultra-low power MCUs,
highlighting their potential for enabling TinyML on resource-constrained
devices. The core of the review centres on efficient neural networks for
TinyML. It covers techniques such as model compression, quantization, and
low-rank factorization, which optimize neural network architectures for minimal
resource utilization on MCUs. The paper then delves into the deployment of deep
learning models on ultra-low power MCUs, addressing challenges such as limited
computational capabilities and memory resources. Techniques like model pruning,
hardware acceleration, and algorithm-architecture co-design are discussed as
strategies to enable efficient deployment. Lastly, the review provides an
overview of current limitations in the field, including the trade-off between
model complexity and resource constraints. Overall, this review paper presents
a comprehensive analysis of efficient neural networks and deployment strategies
for TinyML on ultra-low-power MCUs. It identifies future research directions
for unlocking the full potential of TinyML applications on resource-constrained
devices.