Not Just Novelty: A Longitudinal Study on Utility and Customization of
  AI Workflows
Tao Long Katy Ilonka Gero Lydia B. Chilton
Generative AI brings novel and impressive abilities to help people in
everyday tasks. There are many AI workflows that solve real and complex
problems by chaining AI outputs together with human interaction. Although there
is an undeniable lure of AI, it's uncertain how useful generative AI workflows
are after the novelty wears off. Additionally, tools built with generative AI
have the potential to be personalized and adapted quickly and easily, but do
users take advantage of the potential to customize? We conducted a three-week
longitudinal study with 12 users to understand the familiarization and
customization of generative AI tools for science communication. Our study
revealed that the familiarization phase lasts for 4.3 sessions, where users
explore the capabilities of the workflow and which aspects they find useful.
After familiarization, the perceived utility of the system is rated higher than
before, indicating that the perceived utility of AI is not just a novelty
effect. The increase in benefits mainly comes from end-users' ability to
customize prompts, and thus appropriate the system to their own needs. This
points to a future where generative AI systems can allow us to design for
appropriation.