Enhancing Edge Intelligence with Highly Discriminant LNT Features
Xinyu Wang Vinod K. Mishra C. -C. Jay Kuo
AI algorithms at the edge demand smaller model sizes and lower computational
complexity. To achieve these objectives, we adopt a green learning (GL)
paradigm rather than the deep learning paradigm. GL has three modules: 1)
unsupervised representation learning, 2) supervised feature learning, and 3)
supervised decision learning. We focus on the second module in this work. In
particular, we derive new discriminant features from proper linear combinations
of input features, denoted by x, obtained in the first module. They are called
complementary and raw features, respectively. Along this line, we present a
novel supervised learning method to generate highly discriminant complementary
features based on the least-squares normal transform (LNT). LNT consists of two
steps. First, we convert a C-class classification problem to a binary
classification problem. The two classes are assigned with 0 and 1,
respectively. Next, we formulate a least-squares regression problem from the
N-dimensional (N-D) feature space to the 1-D output space, and solve the
least-squares normal equation to obtain one N-D normal vector, denoted by a1.
Since one normal vector is yielded by one binary split, we can obtain M normal
vectors with M splits. Then, Ax is called an LNT of x, where transform matrix A
in R^{M by N} by stacking aj^T, j=1, ..., M, and the LNT, Ax, can generate M
new features. The newly generated complementary features are shown to be more
discriminant than the raw features. Experiments show that the classification
performance can be improved by these new features.