Can Chat GPT solve a Linguistics Exam?
Patricia Ronan Gerold Schneider
The present study asks if ChatGPT4, the version of ChatGPT which uses the
language model GPT4, can successfully solve introductory linguistic exams.
Previous exam questions of an Introduction to Linguistics course at a German
university are used to test this. The exam questions were fed into ChatGPT4
with only minimal preprocessing. The results show that the language model is
very successful in the interpretation even of complex and nested tasks. It
proved surprisingly successful in the task of broad phonetic transcription, but
performed less well in the analysis of morphemes and phrases. In simple cases
it performs sufficiently well, but rarer cases, particularly with missing
one-to-one correspondence, are currently treated with mixed results. The model
is not yet able to deal with visualisations, such as the analysis or generation
of syntax trees. More extensive preprocessing, which translates these tasks
into text data, allow the model to also solve these tasks successfully.