


The software, called SimSensei and developed by Stefan Scherer and colleagues, is essentially a clever mix of computer vision algorithms and the psychological model of depression. The on-screen psychologist asks you leading questions — a lot like the old-school Eliza, or Alice — and then watches how you physically respond. Using Kinect, the computer vision algorithms build up a very detailed model of your face and body, including your “smile level,” horizontal gaze and vertical gaze, how wide open your eyes are, and whether you are leaning toward or away from the camera. From these markers, SimSensei can work out whether you’re exhibiting signs that indicate depression — gaze aversion, smiling less, and fidgeting. Watch the video below and be amazed.

In testing, the University of Southern California researchers tested a group of 60 people, half of which had been previously diagnosed with depression. SimSensei accurately diagnosed 90% of the people with depression.

What’s most shocking about SimSensei is it illustrates just how easy it is to diagnose depression. If all it takes is a low-res Kinect sensor and some fairly coarse computer vision signals to detect depression, it makes you wonder about how simple the human processes are for detecting depression in other humans. While we might dress it up as empathy or peering into someone’s soul, in actuality our brains might be doing nothing more than looking for gaze aversion and fidgetiness.

It also makes you wonder about how many depressed people go undiagnosed, or worse, wrongly diagnosed. Using an automated, objective system that takes the human element out of the equation would mean that depression could be picked up much sooner, and treated accordingly. It wouldn’t be hard for Microsoft to provide free, regular screening for all Xbox users. And why stop there? If depression can be diagnosed, why not other mood disorders such as bipolar?

By combining Kinect with other sensors, or using a different array of sensors entirely, it might even be possible for a computer to diagnose all sorts of mental and physical issues. Back in 2012, researchers showed that it was possible to detect Parkinson’s disease with a three-minute phone call with a computer; last week we wrote about a sensor that goes under your skin and constantly checks your blood to see if you’re about to have a heart attack (video below).  It is not completely crazy to think that next-generation consoles and smartphones, with the right attachments, will be capable of almost tricorder-like diagnostics.

Now read: Turning the smartphone from a telephone into a tricorder