An unsupervised learning approach to evaluate questionnaire data -- what
  one can learn from violations of measurement invariance
Max Hahn-Klimroth Paul W. Dierkes Matthias W. Kleespies
In several branches of the social sciences and humanities, surveys based on
standardized questionnaires are a prominent research tool. While there are a
variety of ways to analyze the data, some standard procedures have become
established. When those surveys want to analyze differences in the answer
patterns of different groups (e.g., countries, gender, age, ...), these
procedures can only be carried out in a meaningful way if there is measurement
invariance, i.e., the measured construct has psychometric equivalence across
groups. As recently raised as an open problem by Sauerwein et al. (2021), new
evaluation methods that work in the absence of measurement invariance are
needed.
  This paper promotes an unsupervised learning-based approach to such research
data by proposing a procedure that works in three phases: data preparation,
clustering of questionnaires, and measuring similarity based on the obtained
clustering and the properties of each group. We generate synthetic data in
three data sets, which allows us to compare our approach with the PCA approach
under measurement invariance and under violated measurement invariance. As a
main result, we obtain that the approach provides a natural comparison between
groups and a natural description of the response patterns of the groups.
Moreover, it can be safely applied to a wide variety of data sets, even in the
absence of measurement invariance. Finally, this approach allows us to
translate (violations of) measurement invariance into a meaningful measure of
similarity.