Selective imitation on the basis of reward function similarity
Max Taylor-Davies Stephanie Droop Christopher G. Lucas
Imitation is a key component of human social behavior, and is widely used by
both children and adults as a way to navigate uncertain or unfamiliar
situations. But in an environment populated by multiple heterogeneous agents
pursuing different goals or objectives, indiscriminate imitation is unlikely to
be an effective strategy -- the imitator must instead determine who is most
useful to copy. There are likely many factors that play into these judgements,
depending on context and availability of information. Here we investigate the
hypothesis that these decisions involve inferences about other agents' reward
functions. We suggest that people preferentially imitate the behavior of others
they deem to have similar reward functions to their own. We further argue that
these inferences can be made on the basis of very sparse or indirect data, by
leveraging an inductive bias toward positing the existence of different
\textit{groups} or \textit{types} of people with similar reward functions,
allowing learners to select imitation targets without direct evidence of
alignment.