Manipulating Sparse Double Descent
Ya Shi Zhang
This paper investigates the double descent phenomenon in two-layer neural
networks, focusing on the role of L1 regularization and representation
dimensions. It explores an alternative double descent phenomenon, named sparse
double descent. The study emphasizes the complex relationship between model
complexity, sparsity, and generalization, and suggests further research into
more diverse models and datasets. The findings contribute to a deeper
understanding of neural network training and optimization.