Argumentation Element Annotation Modeling using XLNet
Christopher Ormerod Amy Burkhardt Mackenzie Young Sue Lottridge
This study demonstrates the effectiveness of XLNet, a transformer-based
language model, for annotating argumentative elements in persuasive essays.
XLNet's architecture incorporates a recurrent mechanism that allows it to model
long-term dependencies in lengthy texts. Fine-tuned XLNet models were applied
to three datasets annotated with different schemes - a proprietary dataset
using the Annotations for Revisions and Reflections on Writing (ARROW) scheme,
the PERSUADE corpus, and the Argument Annotated Essays (AAE) dataset. The XLNet
models achieved strong performance across all datasets, even surpassing human
agreement levels in some cases. This shows XLNet capably handles diverse
annotation schemes and lengthy essays. Comparisons between the model outputs on
different datasets also revealed insights into the relationships between the
annotation tags. Overall, XLNet's strong performance on modeling argumentative
structures across diverse datasets highlights its suitability for providing
automated feedback on essay organization.