Neural lasso: a unifying approach of lasso and neural networks
David Delgado Ernesto Curbelo Danae Carreras
In recent years, there is a growing interest in combining techniques
attributed to the areas of Statistics and Machine Learning in order to obtain
the benefits of both approaches. In this article, the statistical technique
lasso for variable selection is represented through a neural network. It is
observed that, although both the statistical approach and its neural version
have the same objective function, they differ due to their optimization. In
particular, the neural version is usually optimized in one-step using a single
validation set, while the statistical counterpart uses a two-step optimization
based on cross-validation. The more elaborated optimization of the statistical
method results in more accurate parameter estimation, especially when the
training set is small. For this reason, a modification of the standard approach
for training neural networks, that mimics the statistical framework, is
proposed. During the development of the above modification, a new optimization
algorithm for identifying the significant variables emerged. Experimental
results, using synthetic and real data sets, show that this new optimization
algorithm achieves better performance than any of the three previous
optimization approaches.