Remote Sensing ChatGPT: Solving Remote Sensing Tasks with ChatGPT and
  Visual Models
Haonan Guo Xin Su Chen Wu Bo Du Liangpei Zhang Deren Li
Recently, the flourishing large language models(LLM), especially ChatGPT,
have shown exceptional performance in language understanding, reasoning, and
interaction, attracting users and researchers from multiple fields and domains.
Although LLMs have shown great capacity to perform human-like task
accomplishment in natural language and natural image, their potential in
handling remote sensing interpretation tasks has not yet been fully explored.
Moreover, the lack of automation in remote sensing task planning hinders the
accessibility of remote sensing interpretation techniques, especially to
non-remote sensing experts from multiple research fields. To this end, we
present Remote Sensing ChatGPT, an LLM-powered agent that utilizes ChatGPT to
connect various AI-based remote sensing models to solve complicated
interpretation tasks. More specifically, given a user request and a remote
sensing image, we utilized ChatGPT to understand user requests, perform task
planning according to the tasks' functions, execute each subtask iteratively,
and generate the final response according to the output of each subtask.
Considering that LLM is trained with natural language and is not capable of
directly perceiving visual concepts as contained in remote sensing images, we
designed visual cues that inject visual information into ChatGPT. With Remote
Sensing ChatGPT, users can simply send a remote sensing image with the
corresponding request, and get the interpretation results as well as language
feedback from Remote Sensing ChatGPT. Experiments and examples show that Remote
Sensing ChatGPT can tackle a wide range of remote sensing tasks and can be
extended to more tasks with more sophisticated models such as the remote
sensing foundation model. The code and demo of Remote Sensing ChatGPT is
publicly available at https://github.com/HaonanGuo/Remote-Sensing-ChatGPT .