Predictive Minds: LLMs As Atypical Active Inference Agents
Jan Kulveit Clem von Stengel Roman Leventov
Large language models (LLMs) like GPT are often conceptualized as passive
predictors, simulators, or even stochastic parrots. We instead conceptualize
LLMs by drawing on the theory of active inference originating in cognitive
science and neuroscience. We examine similarities and differences between
traditional active inference systems and LLMs, leading to the conclusion that,
currently, LLMs lack a tight feedback loop between acting in the world and
perceiving the impacts of their actions, but otherwise fit in the active
inference paradigm. We list reasons why this loop may soon be closed, and
possible consequences of this including enhanced model self-awareness and the
drive to minimize prediction error by changing the world.