The pitfalls of next-token prediction
Gregor Bachmann Vaishnavh Nagarajan
Can a mere next-token predictor faithfully model human intelligence? We
crystallize this intuitive concern, which is fragmented in the literature. As a
starting point, we argue that the two often-conflated phases of next-token
prediction -- autoregressive inference and teacher-forced training -- must be
treated distinctly. The popular criticism that errors can compound during
autoregressive inference, crucially assumes that teacher-forcing has learned an
accurate next-token predictor. This assumption sidesteps a more deep-rooted
problem we expose: in certain classes of tasks, teacher-forcing can simply fail
to learn an accurate next-token predictor in the first place. We describe a
general mechanism of how teacher-forcing can fail, and design a minimal
planning task where both the Transformer and the Mamba architecture empirically
fail in that manner -- remarkably, despite the task being straightforward to
learn. We provide preliminary evidence that this failure can be resolved when
training to predict multiple tokens in advance. We hope this finding can ground
future debates and inspire explorations beyond the next-token prediction
paradigm. We make our code available under
https://github.com/gregorbachmann/Next-Token-Failures