Large language models converge toward human-like concept organization
Mathias Lykke Gammelgaard Jonathan Gabel Christiansen Anders Søgaard
Large language models show human-like performance in knowledge extraction,
reasoning and dialogue, but it remains controversial whether this performance
is best explained by memorization and pattern matching, or whether it reflects
human-like inferential semantics and world knowledge. Knowledge bases such as
WikiData provide large-scale, high-quality representations of inferential
semantics and world knowledge. We show that large language models learn to
organize concepts in ways that are strikingly similar to how concepts are
organized in such knowledge bases. Knowledge bases model collective,
institutional knowledge, and large language models seem to induce such
knowledge from raw text. We show that bigger and better models exhibit more
human-like concept organization, across four families of language models and
three knowledge graph embeddings.