AuditGPT: Auditing Smart Contracts with ChatGPT
Shihao Xia Shuai Shao Mengting He Tingting Yu Linhai Song Yiying Zhang
To govern smart contracts running on Ethereum, multiple Ethereum Request for
Comment (ERC) standards have been developed, each containing a set of rules to
guide the behaviors of smart contracts. Violating the ERC rules could cause
serious security issues and financial loss, signifying the importance of
verifying smart contracts follow ERCs. Today's practices of such verification
are to either manually audit each single contract or use expert-developed,
limited-scope program-analysis tools, both of which are far from being
effective in identifying ERC rule violations. This paper presents a tool named
AuditGPT that leverages large language models (LLMs) to automatically and
comprehensively verify ERC rules against smart contracts. To build AuditGPT, we
first conduct an empirical study on 222 ERC rules specified in four popular
ERCs to understand their content, their security impacts, their specification
in natural language, and their implementation in Solidity. Guided by the study,
we construct AuditGPT by separating the large, complex auditing process into
small, manageable tasks and design prompts specialized for each ERC rule type
to enhance LLMs' auditing performance. In the evaluation, AuditGPT successfully
pinpoints 418 ERC rule violations and only reports 18 false positives,
showcasing its effectiveness and accuracy. Moreover, AuditGPT beats an auditing
service provided by security experts in effectiveness, accuracy, and cost,
demonstrating its advancement over state-of-the-art smart-contract auditing
practices.