Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in
  Transformer Models
Steve Yadlowsky Lyric Doshi Nilesh Tripuraneni
Transformer models, notably large language models (LLMs), have the remarkable
ability to perform in-context learning (ICL) -- to perform new tasks when
prompted with unseen input-output examples without any explicit model training.
In this work, we study how effectively transformers can bridge between their
pretraining data mixture, comprised of multiple distinct task families, to
identify and learn new tasks in-context which are both inside and outside the
pretraining distribution. Building on previous work, we investigate this
question in a controlled setting, where we study transformer models trained on
sequences of $(x, f(x))$ pairs rather than natural language. Our empirical
results show transformers demonstrate near-optimal unsupervised model selection
capabilities, in their ability to first in-context identify different task
families and in-context learn within them when the task families are
well-represented in their pretraining data. However when presented with tasks
or functions which are out-of-domain of their pretraining data, we demonstrate
various failure modes of transformers and degradation of their generalization
for even simple extrapolation tasks. Together our results highlight that the
impressive ICL abilities of high-capacity sequence models may be more closely
tied to the coverage of their pretraining data mixtures than inductive biases
that create fundamental generalization capabilities.