Dynamic Early Exiting Predictive Coding Neural Networks
Alaa Zniber Ouassim Karrakchou Mounir Ghogho
Internet of Things (IoT) sensors are nowadays heavily utilized in various
real-world applications ranging from wearables to smart buildings passing by
agrotechnology and health monitoring. With the huge amounts of data generated
by these tiny devices, Deep Learning (DL) models have been extensively used to
enhance them with intelligent processing. However, with the urge for smaller
and more accurate devices, DL models became too heavy to deploy. It is thus
necessary to incorporate the hardware's limited resources in the design
process. Therefore, inspired by the human brain known for its efficiency and
low power consumption, we propose a shallow bidirectional network based on
predictive coding theory and dynamic early exiting for halting further
computations when a performance threshold is surpassed. We achieve comparable
accuracy to VGG-16 in image classification on CIFAR-10 with fewer parameters
and less computational complexity.