Evaluating the Potential of Leading Large Language Models in Reasoning
  Biology Questions
Xinyu Gong Jason Holmes Yiwei Li Zhengliang Liu Qi Gan Zihao Wu Jianli Zhang Yusong Zou Yuxi Teng Tian Jiang Hongtu Zhu Wei Liu Tianming Liu Yajun Yan
Recent advances in Large Language Models (LLMs) have presented new
opportunities for integrating Artificial General Intelligence (AGI) into
biological research and education. This study evaluated the capabilities of
leading LLMs, including GPT-4, GPT-3.5, PaLM2, Claude2, and SenseNova, in
answering conceptual biology questions. The models were tested on a
108-question multiple-choice exam covering biology topics in molecular biology,
biological techniques, metabolic engineering, and synthetic biology. Among the
models, GPT-4 achieved the highest average score of 90 and demonstrated the
greatest consistency across trials with different prompts. The results
indicated GPT-4's proficiency in logical reasoning and its potential to aid
biology research through capabilities like data analysis, hypothesis
generation, and knowledge integration. However, further development and
validation are still required before the promise of LLMs in accelerating
biological discovery can be realized.