A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5
  and Bard AI Models for Java Functions
Giuseppe Destefanis Silvia Bartolucci Marco Ortu
This paper evaluates the capability of two state-of-the-art artificial
intelligence (AI) models, GPT-3.5 and Bard, in generating Java code given a
function description. We sourced the descriptions from CodingBat.com, a popular
online platform that provides practice problems to learn programming. We
compared the Java code generated by both models based on correctness, verified
through the platform's own test cases. The results indicate clear differences
in the capabilities of the two models. GPT-3.5 demonstrated superior
performance, generating correct code for approximately 90.6% of the function
descriptions, whereas Bard produced correct code for 53.1% of the functions.
While both models exhibited strengths and weaknesses, these findings suggest
potential avenues for the development and refinement of more advanced
AI-assisted code generation tools. The study underlines the potential of AI in
automating and supporting aspects of software development, although further
research is required to fully realize this potential.