On-Device Learning with Binary Neural Networks
Lorenzo Vorabbi Davide Maltoni Stefano Santi
Existing Continual Learning (CL) solutions only partially address the
constraints on power, memory and computation of the deep learning models when
deployed on low-power embedded CPUs. In this paper, we propose a CL solution
that embraces the recent advancements in CL field and the efficiency of the
Binary Neural Networks (BNN), that use 1-bit for weights and activations to
efficiently execute deep learning models. We propose a hybrid quantization of
CWR* (an effective CL approach) that considers differently forward and backward
pass in order to retain more precision during gradient update step and at the
same time minimizing the latency overhead. The choice of a binary network as
backbone is essential to meet the constraints of low power devices and, to the
best of authors' knowledge, this is the first attempt to prove on-device
learning with BNN. The experimental validation carried out confirms the
validity and the suitability of the proposed method.