Empathy and the Right to Be an Exception: What LLMs Can and Cannot Do
William Kidder Jason D'Cruz Kush R. Varshney
Advances in the performance of large language models (LLMs) have led some
researchers to propose the emergence of theory of mind (ToM) in artificial
intelligence (AI). LLMs can attribute beliefs, desires, intentions, and
emotions, and they will improve in their accuracy. Rather than employing the
characteristically human method of empathy, they learn to attribute mental
states by recognizing linguistic patterns in a dataset that typically do not
include that individual. We ask whether LLMs' inability to empathize precludes
them from honoring an individual's right to be an exception, that is, from
making assessments of character and predictions of behavior that reflect
appropriate sensitivity to a person's individuality. Can LLMs seriously
consider an individual's claim that their case is different based on internal
mental states like beliefs, desires, and intentions, or are they limited to
judging that case based on its similarities to others? We propose that the
method of empathy has special significance for honoring the right to be an
exception that is distinct from the value of predictive accuracy, at which LLMs
excel. We conclude by considering whether using empathy to consider exceptional
cases has intrinsic or merely practical value and we introduce conceptual and
empirical avenues for advancing this investigation.