ChatGPT Performance on Standardized Testing Exam -- A Proposed Strategy
  for Learners
Umer Farooq Saira Anwar
This study explores the problem solving capabilities of ChatGPT and its
prospective applications in standardized test preparation, focusing on the GRE
quantitative exam. Prior research has shown great potential for the utilization
of ChatGPT for academic purposes in revolutionizing the approach to studying
across various disciplines. We investigate how ChatGPT performs across various
question types in the GRE quantitative domain, and how modifying question
prompts impacts its accuracy. More specifically this study addressed two
research questions: 1. How does ChatGPT perform in answering GRE-based
quantitative questions across various content areas? 2. How does the accuracy
of ChatGPT vary with modifying the question prompts? The dataset consisting of
100 randomly selected GRE quantitative questions was collected from the ETS
official guide to GRE test preparation. We used quantitative evaluation to
answer our first research question, and t-test to examine the statistical
association between prompt modification and ChatGPT's accuracy. Results show a
statistical improvement in the ChatGPT's accuracy after applying instruction
priming and contextual prompts to the original questions. ChatGPT showed 84%
accuracy with the modified prompts compared to 69% with the original data. The
study discusses the areas where ChatGPT struggled with certain questions and
how modifications can be helpful for preparing for standardized tests like GRE
and provides future directions for prompt modifications.