Experimental Narratives: A Comparison of Human Crowdsourced Storytelling
  and AI Storytelling
Nina Begus
The paper proposes a framework that combines behavioral and computational
experiments employing fictional prompts as a novel tool for investigating
cultural artifacts and social biases in storytelling both by humans and
generative AI. The study analyzes 250 stories authored by crowdworkers in June
2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging
methods from narratology and inferential statistics. Both crowdworkers and
large language models responded to identical prompts about creating and falling
in love with an artificial human. The proposed experimental paradigm allows a
direct comparison between human and LLM-generated storytelling. Responses to
the Pygmalionesque prompts confirm the pervasive presence of the Pygmalion myth
in the collective imaginary of both humans and large language models. All
solicited narratives present a scientific or technological pursuit. The
analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more
more progressive in terms of gender roles and sexuality than those written by
humans. While AI narratives can occasionally provide innovative plot twists,
they offer less imaginative scenarios and rhetoric than human-authored texts.
The proposed framework argues that fiction can be used as a window into human
and AI-based collective imaginary and social dimensions.