Scheduled Knowledge Acquisition on Lightweight Vector Symbolic
  Architectures for Brain-Computer Interfaces
Yejia Liu Shijin Duan Xiaolin Xu Shaolei Ren
Brain-Computer interfaces (BCIs) are typically designed to be lightweight and
responsive in real-time to provide users timely feedback. Classical feature
engineering is computationally efficient but has low accuracy, whereas the
recent neural networks (DNNs) improve accuracy but are computationally
expensive and incur high latency. As a promising alternative, the
low-dimensional computing (LDC) classifier based on vector symbolic
architecture (VSA), achieves small model size yet higher accuracy than
classical feature engineering methods. However, its accuracy still lags behind
that of modern DNNs, making it challenging to process complex brain signals. To
improve the accuracy of a small model, knowledge distillation is a popular
method. However, maintaining a constant level of distillation between the
teacher and student models may not be the best way for a growing student during
its progressive learning stages. In this work, we propose a simple scheduled
knowledge distillation method based on curriculum data order to enable the
student to gradually build knowledge from the teacher model, controlled by an
$\alpha$ scheduler. Meanwhile, we employ the LDC/VSA as the student model to
enhance the on-device inference efficiency for tiny BCI devices that demand low
latency. The empirical results have demonstrated that our approach achieves
better tradeoff between accuracy and hardware efficiency compared to other
methods.