Bridging the Digital Divide: Performance Variation across Socio-Economic
  Factors in Vision-Language Models
Joan Nwatu Oana Ignat Rada Mihalcea
Despite the impressive performance of current AI models reported across
various tasks, performance reports often do not include evaluations of how
these models perform on the specific groups that will be impacted by these
technologies. Among the minority groups under-represented in AI, data from
low-income households are often overlooked in data collection and model
evaluation. We evaluate the performance of a state-of-the-art vision-language
model (CLIP) on a geo-diverse dataset containing household images associated
with different income values (Dollar Street) and show that performance
inequality exists among households of different income levels. Our results
indicate that performance for the poorer groups is consistently lower than the
wealthier groups across various topics and countries. We highlight insights
that can help mitigate these issues and propose actionable steps for
economic-level inclusive AI development. Code is available at
https://github.com/MichiganNLP/Bridging_the_Digital_Divide.