LLamol: A Dynamic Multi-Conditional Generative Transformer for De Novo
  Molecular Design
Niklas Dobberstein Astrid Maass Jan Hamaekers
Generative models have demonstrated substantial promise in Natural Language
Processing (NLP) and have found application in designing molecules, as seen in
General Pretrained Transformer (GPT) models. In our efforts to develop such a
tool for exploring the organic chemical space in search of potentially
electro-active compounds, we present "LLamol", a single novel generative
transformer model based on the LLama 2 architecture, which was trained on a 13M
superset of organic compounds drawn from diverse public sources. To allow for a
maximum flexibility in usage and robustness in view of potentially incomplete
data, we introduce "Stochastic Context Learning" as a new training procedure.
We demonstrate that the resulting model adeptly handles single- and
multi-conditional organic molecule generation with up to four conditions, yet
more are possible. The model generates valid molecular structures in SMILES
notation while flexibly incorporating three numerical and/or one token sequence
into the generative process, just as requested. The generated compounds are
very satisfactory in all scenarios tested. In detail, we showcase the model's
capability to utilize token sequences for conditioning, either individually or
in combination with numerical properties, making LLamol a potent tool for de
novo molecule design, easily expandable with new properties.