Prompting Implicit Discourse Relation Annotation
Frances Yung Mansoor Ahmad Merel Scholman Vera Demberg
Pre-trained large language models, such as ChatGPT, archive outstanding
performance in various reasoning tasks without supervised training and were
found to have outperformed crowdsourcing workers. Nonetheless, ChatGPT's
performance in the task of implicit discourse relation classification, prompted
by a standard multiple-choice question, is still far from satisfactory and
considerably inferior to state-of-the-art supervised approaches. This work
investigates several proven prompting techniques to improve ChatGPT's
recognition of discourse relations. In particular, we experimented with
breaking down the classification task that involves numerous abstract labels
into smaller subtasks. Nonetheless, experiment results show that the inference
accuracy hardly changes even with sophisticated prompt engineering, suggesting
that implicit discourse relation classification is not yet resolvable under
zero-shot or few-shot settings.