Scalable variable selection for two-view learning tasks with projection
  operators
Sandor Szedmak Riikka Huusari Tat Hong Duong Le Juho Rousu
In this paper we propose a novel variable selection method for two-view
settings, or for vector-valued supervised learning problems. Our framework is
able to handle extremely large scale selection tasks, where number of data
samples could be even millions. In a nutshell, our method performs variable
selection by iteratively selecting variables that are highly correlated with
the output variables, but which are not correlated with the previously chosen
variables. To measure the correlation, our method uses the concept of
projection operators and their algebra. With the projection operators the
relationship, correlation, between sets of input and output variables can also
be expressed by kernel functions, thus nonlinear correlation models can be
exploited as well. We experimentally validate our approach, showing on both
synthetic and real data its scalability and the relevance of the selected
features. Keywords: Supervised variable selection, vector-valued learning,
projection-valued measure, reproducing kernel Hilbert space