At Google, we’re able to use the large amounts of data made
available by the Web’s fast growth. Two such data sources are
the anonymized queries on google.com and the web itself. They
help improve automatic speech recognition through large language
models: Voice Search makes use of the former, whereas YouTube
speech transcription benefits significantly from the latter.
The language model is the component of a speech recognizer that
assigns a probability to the next word in a sentence given the
previous ones. As an example, if the previous words are “new
york”, the model would assign a higher probability to “pizza”
than say “granola”. The n-gram approach to language modeling
(predicting the next word based on the previous n-1 words) is
particularly well-suited to such large amounts of data: it
scales gracefully, and the non-parametric nature of the model
allows it to grow with more data. For example, on Voice Search
we were able to train and evaluate 5-gram language models
consisting of 12 billion n-grams, bu