Test-takers have a say: understanding the implications of the use of AI
  in language tests
Dawen Zhang Thong Hoang Shidong Pan Yongquan Hu Zhenchang Xing Mark Staples Xiwei Xu Qinghua Lu Aaron Quigley
Language tests measure a person's ability to use a language in terms of
listening, speaking, reading, or writing. Such tests play an integral role in
academic, professional, and immigration domains, with entities such as
educational institutions, professional accreditation bodies, and governments
using them to assess candidate language proficiency. Recent advances in
Artificial Intelligence (AI) and the discipline of Natural Language Processing
have prompted language test providers to explore AI's potential applicability
within language testing, leading to transformative activity patterns
surrounding language instruction and learning. However, with concerns over AI's
trustworthiness, it is imperative to understand the implications of integrating
AI into language testing. This knowledge will enable stakeholders to make
well-informed decisions, thus safeguarding community well-being and testing
integrity. To understand the concerns and effects of AI usage in language
tests, we conducted interviews and surveys with English test-takers. To the
best of our knowledge, this is the first empirical study aimed at identifying
the implications of AI adoption in language tests from a test-taker
perspective. Our study reveals test-taker perceptions and behavioral patterns.
Specifically, we identify that AI integration may enhance perceptions of
fairness, consistency, and availability. Conversely, it might incite mistrust
regarding reliability and interactivity aspects, subsequently influencing the
behaviors and well-being of test-takers. These insights provide a better
understanding of potential societal implications and assist stakeholders in
making informed decisions concerning AI usage in language testing.