A Picture is Worth 500 Labels: A Case Study of Demographic Disparities
  in Local Machine Learning Models for Instagram and TikTok
Jack West Lea Thiemt Shimaa Ahmed Maggie Bartig Kassem Fawaz Suman Banerjee
Mobile apps have embraced user privacy by moving their data processing to the
user's smartphone. Advanced machine learning (ML) models, such as vision
models, can now locally analyze user images to extract insights that drive
several functionalities. Capitalizing on this new processing model of locally
analyzing user images, we analyze two popular social media apps, TikTok and
Instagram, to reveal (1) what insights vision models in both apps infer about
users from their image and video data and (2) whether these models exhibit
performance disparities with respect to demographics. As vision models provide
signals for sensitive technologies like age verification and facial
recognition, understanding potential biases in these models is crucial for
ensuring that users receive equitable and accurate services.
  We develop a novel method for capturing and evaluating ML tasks in mobile
apps, overcoming challenges like code obfuscation, native code execution, and
scalability. Our method comprises ML task detection, ML pipeline
reconstruction, and ML performance assessment, specifically focusing on
demographic disparities. We apply our methodology to TikTok and Instagram,
revealing significant insights. For TikTok, we find issues in age and gender
prediction accuracy, particularly for minors and Black individuals. In
Instagram, our analysis uncovers demographic disparities in the extraction of
over 500 visual concepts from images, with evidence of spurious correlations
between demographic features and certain concepts.