HEAL: Brain-inspired Hyperdimensional Efficient Active Learning
Yang Ni Zhuowen Zou Wenjun Huang Hanning Chen William Youngwoo Chung Samuel Cho Ranganath Krishnan Pietro Mercati Mohsen Imani
Drawing inspiration from the outstanding learning capability of our human
brains, Hyperdimensional Computing (HDC) emerges as a novel computing paradigm,
and it leverages high-dimensional vector presentation and operations for
brain-like lightweight Machine Learning (ML). Practical deployments of HDC have
significantly enhanced the learning efficiency compared to current deep ML
methods on a broad spectrum of applications. However, boosting the data
efficiency of HDC classifiers in supervised learning remains an open question.
In this paper, we introduce Hyperdimensional Efficient Active Learning (HEAL),
a novel Active Learning (AL) framework tailored for HDC classification. HEAL
proactively annotates unlabeled data points via uncertainty and
diversity-guided acquisition, leading to a more efficient dataset annotation
and lowering labor costs. Unlike conventional AL methods that only support
classifiers built upon deep neural networks (DNN), HEAL operates without the
need for gradient or probabilistic computations. This allows it to be
effortlessly integrated with any existing HDC classifier architecture. The key
design of HEAL is a novel approach for uncertainty estimation in HDC
classifiers through a lightweight HDC ensemble with prior hypervectors.
Additionally, by exploiting hypervectors as prototypes (i.e., compact
representations), we develop an extra metric for HEAL to select diverse samples
within each batch for annotation. Our evaluation shows that HEAL surpasses a
diverse set of baselines in AL quality and achieves notably faster acquisition
than many BNN-powered or diversity-guided AL methods, recording 11 times to
40,000 times speedup in acquisition runtime per batch.