Short Answer Grading Using One-shot Prompting and Text Similarity
  Scoring Model
Su-Youn Yoon
In this study, we developed an automated short answer grading (ASAG) model
that provided both analytic scores and final holistic scores. Short answer
items typically consist of multiple sub-questions, and providing an analytic
score and the text span relevant to each sub-question can increase the
interpretability of the automated scores. Furthermore, they can be used to
generate actionable feedback for students. Despite these advantages, most
studies have focused on predicting only holistic scores due to the difficulty
in constructing dataset with manual annotations. To address this difficulty, we
used large language model (LLM)-based one-shot prompting and a text similarity
scoring model with domain adaptation using small manually annotated dataset.
The accuracy and quadratic weighted kappa of our model were 0.67 and 0.71 on a
subset of the publicly available ASAG dataset. The model achieved a substantial
improvement over the majority baseline.