Learning to Plan for Language Modeling from Unlabeled Data
Nathan Cornille Marie-Francine Moens Florian Mai
By training to predict the next token in an unlabeled corpus, large language
models learn to perform many tasks without any labeled data. However, their
next-token-prediction objective arguably limits their performance in scenarios
that require planning, such as writing a coherent article. In this paper, we
train a module for planning the future writing process via a self-supervised
learning objective. By conditioning on generated latent plans, our model
extends the successful language model formula to more abstract planning in an
unsupervised way. Empirically, we demonstrate that our method improves language
modeling performance in general, particularly with respect to the text
structure. Because our framework uses a planner module that is unsupervised and
external to the language model, new planner modules can be trained at large
scale and easily be shared with the community.