Learning representations of learning representations
Rita González-Márquez Dmitry Kobak
The ICLR conference is unique among the top machine learning conferences in
that all submitted papers are openly available. Here we present the ICLR
dataset consisting of abstracts of all 24 thousand ICLR submissions from
2017-2024 with meta-data, decision scores, and custom keyword-based labels. We
find that on this dataset, bag-of-words representation outperforms most
dedicated sentence transformer models in terms of $k$NN classification
accuracy, and the top performing language models barely outperform TF-IDF. We
see this as a challenge for the NLP community. Furthermore, we use the ICLR
dataset to study how the field of machine learning has changed over the last
seven years, finding some improvement in gender balance. Using a 2D embedding
of the abstracts' texts, we describe a shift in research topics from 2017 to
2024 and identify hedgehogs and foxes among the authors with the highest number
of ICLR submissions.