Embedded Multi-label Feature Selection via Orthogonal Regression
Xueyuan Xu Fulin Wei Tianyuan Jia Li Zhuo Feiping Nie Xia Wu
In the last decade, embedded multi-label feature selection methods,
incorporating the search for feature subsets into model optimization, have
attracted considerable attention in accurately evaluating the importance of
features in multi-label classification tasks. Nevertheless, the
state-of-the-art embedded multi-label feature selection algorithms based on
least square regression usually cannot preserve sufficient discriminative
information in multi-label data. To tackle the aforementioned challenge, a
novel embedded multi-label feature selection method, termed global redundancy
and relevance optimization in orthogonal regression (GRROOR), is proposed to
facilitate the multi-label feature selection. The method employs orthogonal
regression with feature weighting to retain sufficient statistical and
structural information related to local label correlations of the multi-label
data in the feature learning process. Additionally, both global feature
redundancy and global label relevancy information have been considered in the
orthogonal regression model, which could contribute to the search for
discriminative and non-redundant feature subsets in the multi-label data. The
cost function of GRROOR is an unbalanced orthogonal Procrustes problem on the
Stiefel manifold. A simple yet effective scheme is utilized to obtain an
optimal solution. Extensive experimental results on ten multi-label data sets
demonstrate the effectiveness of GRROOR.