The flow of ideas in word embeddings
Debayan Dasgupta
The flow of ideas has been extensively studied by physicists, psychologists,
and machine learning engineers. This paper adopts specific tools from
microrheology to investigate the similarity-based flow of ideas. We introduce a
random walker in word embeddings and study its behavior. Such
similarity-mediated random walks through the embedding space show signatures of
anomalous diffusion commonly observed in complex structured systems such as
biological cells and complex fluids. The paper concludes by proposing the
application of popular tools employed in the study of random walks and
diffusion of particles under Brownian motion to assess quantitatively the
incorporation of diverse ideas in a document. Overall, this paper presents a
self-referenced method combining microrheology and machine learning concepts to
explore the meandering tendencies of language models and their potential
association with creativity.