Accounting for AI and Users Shaping One Another: The Role of
  Mathematical Models
Sarah Dean Evan Dong Meena Jagadeesan Liu Leqi
As AI systems enter into a growing number of societal domains, these systems
increasingly shape and are shaped by user preferences, opinions, and behaviors.
However, the design of AI systems rarely accounts for how AI and users shape
one another. In this position paper, we argue for the development of formal
interaction models which mathematically specify how AI and users shape one
another. Formal interaction models can be leveraged to (1) specify interactions
for implementation, (2) monitor interactions through empirical analysis, (3)
anticipate societal impacts via counterfactual analysis, and (4) control
societal impacts via interventions. The design space of formal interaction
models is vast, and model design requires careful consideration of factors such
as style, granularity, mathematical complexity, and measurability. Using
content recommender systems as a case study, we critically examine the nascent
literature of formal interaction models with respect to these use-cases and
design axes. More broadly, we call for the community to leverage formal
interaction models when designing, evaluating, or auditing any AI system which
interacts with users.