ChatGPT: A Study on its Utility for Ubiquitous Software Engineering
  Tasks
Giriprasad Sridhara Ranjani H. G. Sourav Mazumdar
ChatGPT (Chat Generative Pre-trained Transformer) is a chatbot launched by
OpenAI on November 30, 2022. OpenAI's GPT-3 family of large language models
serve as the foundation for ChatGPT. ChatGPT is fine-tuned with both supervised
and reinforcement learning techniques and has received widespread attention for
its articulate responses across diverse domains of knowledge. In this study, we
explore how ChatGPT can be used to help with common software engineering tasks.
Many of the ubiquitous tasks covering the breadth of software engineering such
as ambiguity resolution in software requirements, method name suggestion, test
case prioritization, code review, log summarization can potentially be
performed using ChatGPT. In this study, we explore fifteen common software
engineering tasks using ChatGPT. We juxtapose and analyze ChatGPT's answers
with the respective state of the art outputs (where available) and/or human
expert ground truth. Our experiments suggest that for many tasks, ChatGPT does
perform credibly and the response from it is detailed and often better than the
human expert output or the state of the art output. However, for a few other
tasks, ChatGPT in its present form provides incorrect answers and hence is not
suited for such tasks.