Future Language Modeling from Temporal Document History
Changmao Li Jeffrey Flanigan
Predicting the future is of great interest across many aspects of human
activity. Businesses are interested in future trends, traders are interested in
future stock prices, and companies are highly interested in future
technological breakthroughs. While there are many automated systems for
predicting future numerical data, such as weather, stock prices, and demand for
products, there is relatively little work in automatically predicting textual
data. Humans are interested in textual data predictions because it is a natural
format for our consumption, and experts routinely make predictions in a textual
format (Christensen et al., 2004; Tetlock & Gardner, 2015; Frick, 2015).
However, there has been relatively little formalization of this general problem
in the machine learning or natural language processing communities. To address
this gap, we introduce the task of future language modeling: probabilistic
modeling of texts in the future based on a temporal history of texts. To our
knowledge, our work is the first work to formalize the task of predicting the
future in this way. We show that it is indeed possible to build future language
models that improve upon strong non-temporal language model baselines, opening
the door to working on this important, and widely applicable problem.