Update README.md

README.md

@@ -1,3 +1,84 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- chemistry
+---
+
+# GP-MoLFormer-Uniq
+
+GP-MoLFormer is a class of models pretrained on SMILES string representations of 0.65-1.1B molecules from ZINC and PubChem.
+This repository is for the model pretrained on all the _unique_ molecules from both datasets.
+
+It was introduced in the paper [GP-MoLFormer: A Foundation Model For Molecular Generation](https://arxiv.org/abs/2405.04912) by Ross et al. and released in [this repository](https://github.com/IBM/gp-molformer).
+
+## Model Details
+
+### Model Description
+
+GP-MoLFormer is a large-scale autoregressive chemical language model intended for molecule generation tasks. GP-MoLFormer employs the same architecture as MoLFormer-XL, including linear attention and rotary position embeddings, but uses decoder-only Transformer blocks trained with a causal language modeling objective. It is trained on up to 1.1B molecules in SMILES representation.
+
+GP-MoLFormer was evaluated on _de novo_ generation (*at scale*), scaffold-constrained decoration, and molecular property optimization tasks.
+
+## Intended use and limitations
+
+The pretrained model may be used out-of-the-box for unconditional, _de novo_ molecule generation. It can also be prompted with a partial SMILES string to do scaffold completion/decoration. We also demonstrate it can be fine-tuned on a particular dataset to change the output distribution (e.g., more druglike) or tuned for molecular optimization using **pair-tuning**. For details, see the paper and GitHub repository.
+
+This model is not tested for classification performance. It is also not tested for molecules larger than ~200 atoms (i.e., macromolecules). Furthermore, using invalid or noncanonical SMILES may result in worse performance.
+
+## Example code
+
+Use the code below to get started with the model.
+
+```py
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("ibm-research/GP-MoLFormer-Uniq", trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained("ibm-research/MoLFormer-XL-both-10pct", trust_remote_code=True)
+
+outputs = model.generate(do_sample=True, top_k=None, max_length=202, num_return_sequences=3)
+tokenizer.batch_decode(outputs, skip_special_tokens=True)
+```
+
+## Training Details
+
+### Data
+
+We trained GP-MoLFormer on a combination of molecules from the ZINC15 and PubChem datasets. This repository contains the version trained on all _unique_ molecules from both datasets.
+
+Molecules were canonicalized with RDKit prior to training and isomeric information was removed. Also, molecules longer than 202 tokens were dropped.
+
+### Hardware
+
+- 16 x NVIDIA A100 80GB GPUs
+
+## Evaluation
+
+We evaluated GP-MoLFormer on various generation metrics. The tables below show the performance of GP-MoLFormer-Uniq compared to baseline models:
+
+|                   | Val↑ | Uniq@10k↑ | Nov↑ | Frag↑ | Scaf↑ | SNN↑ | IntDiv↑ | FCD↓ |
+|-------------------|------------|-----------------|------------|-------------|-------------|------------|---------------|------------|
+| CharRNN           | 0.975      | 0.999           | 0.842      | **0.9998**  | 0.9242      | 0.6015     | 0.8562        | 0.0732     |
+| VAE               | 0.977      | 0.998           | 0.695      | 0.9984      | **0.9386**  | **0.6257** | 0.8558        | 0.0990     |
+| JT-VAE            | **1.000**  | **1.000**       | 0.914      | 0.9965      | 0.8964      | 0.5477     | 0.8551        | 0.3954     |
+| LIMO              | **1.000**  | 0.976           | **1.000**  | 0.6989      | 0.0079      | 0.2464     | **0.9039**    | 26.78      |
+| MolGen-7B         | **1.000**  | **1.000**       | 0.934      | **0.9999**  | 0.6538      | 0.5138     | 0.8617        | **0.0435** |
+| GP-MoLFormer-Uniq | **1.000**  | 0.977           | 0.390      | **0.9998**  | 0.7383      | 0.5045     | 0.8655        | 0.0591     |
+
+We report all metrics using the typical MOSES definitions on each model's respective test set. Note: novelty is with respect to each model's respective training set.
+
+## Citation
+
+```
+@misc{ross2025gpmolformerfoundationmodelmolecular,
+      title={GP-MoLFormer: A Foundation Model For Molecular Generation}, 
+      author={Jerret Ross and Brian Belgodere and Samuel C. Hoffman and Vijil Chenthamarakshan and Jiri Navratil and Youssef Mroueh and Payel Das},
+      year={2025},
+      eprint={2405.04912},
+      archivePrefix={arXiv},
+      primaryClass={q-bio.BM},
+      url={https://arxiv.org/abs/2405.04912}, 
+}
+```