A Hybrid Intelligence Method for Argument Mining
Michiel van der Meer Enrico Liscio Catholijn M. Jonker Aske Plaat Piek Vossen Pradeep K. Murukannaiah
Large-scale survey tools enable the collection of citizen feedback in opinion
corpora. Extracting the key arguments from a large and noisy set of opinions
helps in understanding the opinions quickly and accurately. Fully automated
methods can extract arguments but (1) require large labeled datasets that
induce large annotation costs and (2) work well for known viewpoints, but not
for novel points of view. We propose HyEnA, a hybrid (human + AI) method for
extracting arguments from opinionated texts, combining the speed of automated
processing with the understanding and reasoning capabilities of humans. We
evaluate HyEnA on three citizen feedback corpora. We find that, on the one
hand, HyEnA achieves higher coverage and precision than a state-of-the-art
automated method when compared to a common set of diverse opinions, justifying
the need for human insight. On the other hand, HyEnA requires less human effort
and does not compromise quality compared to (fully manual) expert analysis,
demonstrating the benefit of combining human and artificial intelligence.