Using language models in the implicit automated assessment of
  mathematical short answer items
Christopher Ormerod
We propose a new way to assess certain short constructed responses to
mathematics items. Our approach uses a pipeline that identifies the key values
specified by the student in their response. This allows us to determine the
correctness of the response, as well as identify any misconceptions. The
information from the value identification pipeline can then be used to provide
feedback to the teacher and student. The value identification pipeline consists
of two fine-tuned language models. The first model determines if a value is
implicit in the student response. The second model identifies where in the
response the key value is specified. We consider both a generic model that can
be used for any prompt and value, as well as models that are specific to each
prompt and value. The value identification pipeline is a more accurate and
informative way to assess short constructed responses than traditional
rubric-based scoring. It can be used to provide more targeted feedback to
students, which can help them improve their understanding of mathematics.