ChemDFM: Dialogue Foundation Model for Chemistry
Zihan Zhao Da Ma Lu Chen Liangtai Sun Zihao Li Hongshen Xu Zichen Zhu Su Zhu Shuai Fan Guodong Shen Xin Chen Kai Yu
Large language models (LLMs) have established great success in the general
domain of natural language processing. Their emerging task generalization and
free-form dialogue capabilities can greatly help to design Chemical General
Intelligence (CGI) to assist real-world research in chemistry. However, the
existence of specialized language and knowledge in the field of chemistry, such
as the highly informative SMILES notation, hinders the performance of
general-domain LLMs in chemistry. To this end, we develop ChemDFM, the first
LLM towards CGI. ChemDFM-13B is trained on 34B tokens from chemical literature,
textbooks, and instructions as well as various data from the general domain.
Therefore, it can store, understand, and reason over chemical knowledge and
languages while still possessing advanced free-form language comprehension
capabilities. Extensive quantitative evaluation shows that ChemDFM can
significantly outperform the representative open-sourced LLMs. Moreover,
ChemDFM can also surpass GPT-4 on a great portion of chemical tasks, despite
the significant size difference. Further qualitative evaluations demonstrate
the efficiency and effectiveness of ChemDFM in real-world research scenarios.
We will open-source the ChemDFM model soon.