Automated Feedback Generation for a Chemistry Database and Abstracting
  Exercise
Oscar Morris Russell Morris
Timely feedback is an important part of teaching and learning. Here we
describe how a readily available neural network transformer (machine-learning)
model (BERT) can be used to give feedback on the structure of the response to
an abstracting exercise where students are asked to summarise the contents of a
published article after finding it from a publication database. The dataset
contained 207 submissions from two consecutive years of the course, summarising
a total of 21 different papers from the primary literature. The model was
pre-trained using an available dataset (approx. 15,000 samples) and then
fine-tuned on 80% of the submitted dataset. This fine tuning was seen to be
important. The sentences in the student submissions are characterised into
three classes - background, technique and observation - which allows a
comparison of how each submission is structured. Comparing the structure of the
students' abstract a large collection of those from the PubMed database shows
that students in this exercise concentrate more on the background to the paper
and less on the techniques and results than the abstracts to papers themselves.
The results allowed feedback for each submitted assignment to be automatically
generated.