Harnessing Dataset Cartography for Improved Compositional Generalization
  in Transformers
Osman Batur İnce Tanin Zeraati Semih Yagcioglu Yadollah Yaghoobzadeh Erkut Erdem Aykut Erdem
Neural networks have revolutionized language modeling and excelled in various
downstream tasks. However, the extent to which these models achieve
compositional generalization comparable to human cognitive abilities remains a
topic of debate. While existing approaches in the field have mainly focused on
novel architectures and alternative learning paradigms, we introduce a
pioneering method harnessing the power of dataset cartography (Swayamdipta et
al., 2020). By strategically identifying a subset of compositional
generalization data using this approach, we achieve a remarkable improvement in
model accuracy, yielding enhancements of up to 10% on CFQ and COGS datasets.
Notably, our technique incorporates dataset cartography as a curriculum
learning criterion, eliminating the need for hyperparameter tuning while
consistently achieving superior performance. Our findings highlight the
untapped potential of dataset cartography in unleashing the full capabilities
of compositional generalization within Transformer models. Our code is
available at https://github.com/cyberiada/cartography-for-compositionality.