Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate
  Models for IRT Assessment
Jingshen Zhang Jiajun Xie Xinying Qiu
Item difficulty plays a crucial role in adaptive testing. However, few works
have focused on generating questions of varying difficulty levels, especially
for multiple-choice (MC) cloze tests. We propose training pre-trained language
models (PLMs) as surrogate models to enable item response theory (IRT)
assessment, avoiding the need for human test subjects. We also propose two
strategies to control the difficulty levels of both the gaps and the
distractors using ranking rules to reduce invalid distractors. Experimentation
on a benchmark dataset demonstrates that our proposed framework and methods can
effectively control and evaluate the difficulty levels of MC cloze tests.