Transformers and Large Language Models for Chemistry and Drug Discovery
Andres M Bran Philippe Schwaller
Language modeling has seen impressive progress over the last years, mainly
prompted by the invention of the Transformer architecture, sparking a
revolution in many fields of machine learning, with breakthroughs in chemistry
and biology. In this chapter, we explore how analogies between chemical and
natural language have inspired the use of Transformers to tackle important
bottlenecks in the drug discovery process, such as retrosynthetic planning and
chemical space exploration. The revolution started with models able to perform
particular tasks with a single type of data, like linearised molecular graphs,
which then evolved to include other types of data, like spectra from analytical
instruments, synthesis actions, and human language. A new trend leverages
recent developments in large language models, giving rise to a wave of models
capable of solving generic tasks in chemistry, all facilitated by the
flexibility of natural language. As we continue to explore and harness these
capabilities, we can look forward to a future where machine learning plays an
even more integral role in accelerating scientific discovery.