MoodCapture: Depression Detection Using In-the-Wild Smartphone Images
Subigya Nepal Arvind Pillai Weichen Wang Tess Griffin Amanda C. Collins Michael Heinz Damien Lekkas Shayan Mirjafari Matthew Nemesure George Price Nicholas C. Jacobson Andrew T. Campbell
MoodCapture presents a novel approach that assesses depression based on
images automatically captured from the front-facing camera of smartphones as
people go about their daily lives. We collect over 125,000 photos in the wild
from N=177 participants diagnosed with major depressive disorder for 90 days.
Images are captured naturalistically while participants respond to the PHQ-8
depression survey question: \textit{``I have felt down, depressed, or
hopeless''}. Our analysis explores important image attributes, such as angle,
dominant colors, location, objects, and lighting. We show that a random forest
trained with face landmarks can classify samples as depressed or non-depressed
and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides
several insights through an ablation study, feature importance analysis, and
bias assessment. Importantly, we evaluate user concerns about using MoodCapture
to detect depression based on sharing photos, providing critical insights into
privacy concerns that inform the future design of in-the-wild image-based
mental health assessment tools.