Tree-GPT: Modular Large Language Model Expert System for Forest Remote
  Sensing Image Understanding and Interactive Analysis
Siqi Du Shengjun Tang Weixi Wang Xiaoming Li Renzhong Guo
This paper introduces a novel framework, Tree-GPT, which incorporates Large
Language Models (LLMs) into the forestry remote sensing data workflow, thereby
enhancing the efficiency of data analysis. Currently, LLMs are unable to
extract or comprehend information from images and may generate inaccurate text
due to a lack of domain knowledge, limiting their use in forestry data
analysis. To address this issue, we propose a modular LLM expert system,
Tree-GPT, that integrates image understanding modules, domain knowledge bases,
and toolchains. This empowers LLMs with the ability to comprehend images,
acquire accurate knowledge, generate code, and perform data analysis in a local
environment. Specifically, the image understanding module extracts structured
information from forest remote sensing images by utilizing automatic or
interactive generation of prompts to guide the Segment Anything Model (SAM) in
generating and selecting optimal tree segmentation results. The system then
calculates tree structural parameters based on these results and stores them in
a database. Upon receiving a specific natural language instruction, the LLM
generates code based on a thought chain to accomplish the analysis task. The
code is then executed by an LLM agent in a local environment and . For
ecological parameter calculations, the system retrieves the corresponding
knowledge from the knowledge base and inputs it into the LLM to guide the
generation of accurate code. We tested this system on several tasks, including
Search, Visualization, and Machine Learning Analysis. The prototype system
performed well, demonstrating the potential for dynamic usage of LLMs in
forestry research and environmental sciences.