HPC-GPT: Integrating Large Language Model for High-Performance Computing
Xianzhong Ding Le Chen Murali Emani Chunhua Liao Pei-Hung Lin Tristan Vanderbruggen Zhen Xie Alberto E. Cerpa Wan Du
Large Language Models (LLMs), including the LLaMA model, have exhibited their
efficacy across various general-domain natural language processing (NLP) tasks.
However, their performance in high-performance computing (HPC) domain tasks has
been less than optimal due to the specialized expertise required to interpret
the model responses. In response to this challenge, we propose HPC-GPT, a novel
LLaMA-based model that has been supervised fine-tuning using generated QA
(Question-Answer) instances for the HPC domain. To evaluate its effectiveness,
we concentrate on two HPC tasks: managing AI models and datasets for HPC, and
data race detection. By employing HPC-GPT, we demonstrate comparable
performance with existing methods on both tasks, exemplifying its excellence in
HPC-related scenarios. Our experiments on open-source benchmarks yield
extensive results, underscoring HPC-GPT's potential to bridge the performance
gap between LLMs and HPC-specific tasks. With HPC-GPT, we aim to pave the way
for LLMs to excel in HPC domains, simplifying the utilization of language
models in complex computing applications.