How Well Can You Articulate that Idea? Insights from Automated Formative
  Assessment
Mahsa Sheikhi Karizaki Dana Gnesdilow Sadhana Puntambekar Rebecca J. Passonneau
Automated methods are becoming increasingly integrated into studies of
formative feedback on students' science explanation writing. Most of this work,
however, addresses students' responses to short answer questions. We
investigate automated feedback on students' science explanation essays, where
students must articulate multiple ideas. Feedback is based on a rubric that
identifies the main ideas students are prompted to include in explanatory
essays about the physics of energy and mass, given their experiments with a
simulated roller coaster. We have found that students generally improve on
revised versions of their essays. Here, however, we focus on two factors that
affect the accuracy of the automated feedback. First, we find that the main
ideas in the rubric differ with respect to how much freedom they afford in
explanations of the idea, thus explanation of a natural law is relatively
constrained. Students have more freedom in how they explain complex relations
they observe in their roller coasters, such as transfer of different forms of
energy. Second, by tracing the automated decision process, we can diagnose when
a student's statement lacks sufficient clarity for the automated tool to
associate it more strongly with one of the main ideas above all others. This in
turn provides an opportunity for teachers and peers to help students reflect on
how to state their ideas more clearly.