Social Intelligence Data Infrastructure: Structuring the Present and
  Navigating the Future
Minzhi Li Weiyan Shi Caleb Ziems Diyi Yang
As Natural Language Processing (NLP) systems become increasingly integrated
into human social life, these technologies will need to increasingly rely on
social intelligence. Although there are many valuable datasets that benchmark
isolated dimensions of social intelligence, there does not yet exist any body
of work to join these threads into a cohesive subfield in which researchers can
quickly identify research gaps and future directions. Towards this goal, we
build a Social AI Data Infrastructure, which consists of a comprehensive social
AI taxonomy and a data library of 480 NLP datasets. Our infrastructure allows
us to analyze existing dataset efforts, and also evaluate language models'
performance in different social intelligence aspects. Our analyses demonstrate
its utility in enabling a thorough understanding of current data landscape and
providing a holistic perspective on potential directions for future dataset
development. We show there is a need for multifaceted datasets, increased
diversity in language and culture, more long-tailed social situations, and more
interactive data in future social intelligence data efforts.