Octopus: On-device language model for function calling of software APIs
Wei Chen Zhiyuan Li Mingyuan Ma
In the rapidly evolving domain of artificial intelligence, Large Language
Models (LLMs) play a crucial role due to their advanced text processing and
generation abilities. This study introduces a new strategy aimed at harnessing
on-device LLMs in invoking software APIs. We meticulously compile a dataset
derived from software API documentation and apply fine-tuning to LLMs with
capacities of 2B, 3B and 7B parameters, specifically to enhance their
proficiency in software API interactions. Our approach concentrates on refining
the models' grasp of API structures and syntax, significantly enhancing the
accuracy of API function calls. Additionally, we propose \textit{conditional
masking} techniques to ensure outputs in the desired formats and reduce error
rates while maintaining inference speeds. We also propose a novel benchmark
designed to evaluate the effectiveness of LLMs in API interactions,
establishing a foundation for subsequent research. Octopus, the fine-tuned
model, is proved to have better performance than GPT-4 for the software APIs
calling. This research aims to advance automated software development and API
integration, representing substantial progress in aligning LLM capabilities
with the demands of practical software engineering applications.