Unified Uncertainty Estimation for Cognitive Diagnosis Models
Fei Wang Qi Liu Enhong Chen Chuanren Liu Zhenya Huang Jinze Wu Shijin Wang
Cognitive diagnosis models have been widely used in different areas,
especially intelligent education, to measure users' proficiency levels on
knowledge concepts, based on which users can get personalized instructions. As
the measurement is not always reliable due to the weak links of the models and
data, the uncertainty of measurement also offers important information for
decisions. However, the research on the uncertainty estimation lags behind that
on advanced model structures for cognitive diagnosis. Existing approaches have
limited efficiency and leave an academic blank for sophisticated models which
have interaction function parameters (e.g., deep learning-based models). To
address these problems, we propose a unified uncertainty estimation approach
for a wide range of cognitive diagnosis models. Specifically, based on the idea
of estimating the posterior distributions of cognitive diagnosis model
parameters, we first provide a unified objective function for mini-batch based
optimization that can be more efficiently applied to a wide range of models and
large datasets. Then, we modify the reparameterization approach in order to
adapt to parameters defined on different domains. Furthermore, we decompose the
uncertainty of diagnostic parameters into data aspect and model aspect, which
better explains the source of uncertainty. Extensive experiments demonstrate
that our method is effective and can provide useful insights into the
uncertainty of cognitive diagnosis.