Evolutionary Neural Architecture Search for Transformer in Knowledge
  Tracing
Shangshang Yang Xiaoshan Yu Ye Tian Xueming Yan Haiping Ma Xingyi Zhang
Knowledge tracing (KT) aims to trace students' knowledge states by predicting
whether students answer correctly on exercises. Despite the excellent
performance of existing Transformer-based KT approaches, they are criticized
for the manually selected input features for fusion and the defect of single
global context modelling to directly capture students' forgetting behavior in
KT, when the related records are distant from the current record in terms of
time. To address the issues, this paper first considers adding convolution
operations to the Transformer to enhance its local context modelling ability
used for students' forgetting behavior, then proposes an evolutionary neural
architecture search approach to automate the input feature selection and
automatically determine where to apply which operation for achieving the
balancing of the local/global context modelling. In the search space, the
original global path containing the attention module in Transformer is replaced
with the sum of a global path and a local path that could contain different
convolutions, and the selection of input features is also considered. To search
the best architecture, we employ an effective evolutionary algorithm to explore
the search space and also suggest a search space reduction strategy to
accelerate the convergence of the algorithm. Experimental results on the two
largest and most challenging education datasets demonstrate the effectiveness
of the architecture found by the proposed approach.