Large Language Models Can Infer Psychological Dispositions of Social
  Media Users
Heinrich Peters Sandra Matz
As Large Language Models (LLMs) demonstrate increasingly human-like abilities
in various natural language processing (NLP) tasks that are bound to become
integral to personalized technologies, understanding their capabilities and
inherent biases is crucial. Our study investigates the potential of LLMs like
ChatGPT to infer psychological dispositions of individuals from their digital
footprints. Specifically, we assess the ability of GPT-3.5 and GPT-4 to derive
the Big Five personality traits from users' Facebook status updates in a
zero-shot learning scenario. Our results show an average correlation of r = .29
(range = [.22, .33]) between LLM-inferred and self-reported trait scores.
Furthermore, our findings suggest biases in personality inferences with regard
to gender and age: inferred scores demonstrated smaller errors for women and
younger individuals on several traits, suggesting a potential systematic bias
stemming from the underlying training data or differences in online
self-expression.