A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students'
  Formative Assessment Responses in Science
Clayton Cohn Nicole Hutchins Tuan Le Gautam Biswas
This paper explores the use of large language models (LLMs) to score and
explain short-answer assessments in K-12 science. While existing methods can
score more structured math and computer science assessments, they often do not
provide explanations for the scores. Our study focuses on employing GPT-4 for
automated assessment in middle school Earth Science, combining few-shot and
active learning with chain-of-thought reasoning. Using a human-in-the-loop
approach, we successfully score and provide meaningful explanations for
formative assessment responses. A systematic analysis of our method's pros and
cons sheds light on the potential for human-in-the-loop techniques to enhance
automated grading for open-ended science assessments.