Skip to main content

Human Cues to Improve Computer User Friendliness

  • Published: 2011-03-05 : Binghamton University.
  • Synopsis: Research in computer graphics and computer vision tries to make using computers easier.

Main Document


"Our research in computer graphics and computer vision tries to make using computers easier," says the Binghamton University computer scientist. "Can we find a more comfortable, intuitive and intelligent way to use the computer? It should feel like you're talking to a friend. This could also help disabled people use computers the way everyone else does."

Yin's team has developed ways to provide information to the computer based on where a user is looking as well as through gestures or speech. One of the basic challenges in this area is "computer vision." That is, how can a simple webcam work more like the human eye? Can camera-captured data understand a real-world object? Can this data be used to "see" the user and "understand" what the user wants to do

To some extent, that's already possible. Witness one of Yin's graduate students giving a PowerPoint presentation and using only his eyes to highlight content on various slides. When Yin demonstrated this technology for Air Force experts last year, the only hardware he brought was a webcam attached to a laptop computer.

Yin says the next step would be enabling the computer to recognize a user's emotional state. He works with a well-established set of six basic emotions "anger, disgust, fear, joy, sadness, and surprise "and is experimenting with different ways to allow the computer to distinguish among them. Is there enough data in the way the lines around the eyes change? Could focusing on the user's mouth provide sufficient clues? What happens if the user's face is only partially visible, perhaps turned to one side

"Computers only understand zeroes and ones," Yin says. "Everything is about patterns. We want to find out how to recognize each emotion using only the most important features."

He's partnering with Binghamton University psychologist Peter Gerhardstein to explore ways this work could benefit children with autism. Many people with autism have difficulty interpreting others' emotions; therapists sometimes use photographs of people to teach children how to understand when someone is happy or sad and so forth. Yin could produce not just photographs, but three-dimensional avatars that are able to display a range of emotions. Given the right pictures, Yin could even produce avatars of people from a child's family for use in this type of therapy.



"We want not only to create a virtual-person model, we want to understand a real person's emotions and feelings," Yin says. "We want the computer to be able to understand how you feel, too. That's hard, even harder than my other work."

Imagine if a computer could understand when people are in pain. Some may ask a doctor for help. But others "young children, for instance "cannot express themselves or are unable to speak for some reason. Yin wants to develop an algorithm that would enable a computer to determine when someone is in pain based just on a photograph.


"This technology," Yin says, "could help us to train the computer to do facial-recognition analysis in place of experts."


A related video clip is available at

Similar Topics

1 - Virtual Reality Goes Magnetic with Electronic 'Skin' Sensors - Helmholtz-Zentrum Dresden-Rossendorf.
2 - JAMBOXX an NYSID Adaptive technology Product - Tom Occhiogrosso NYSID.
3 - Athena Fund Distributes iPads to 3,000 Special Education and Kindergarten Teachers in Israel in 2017 - Athena Fund.
4 - Foot Mouse with Programmable Pedal - Disabled World.
5 - Israeli President Honors Special Education Teachers - Athena Fund.
From our Electronic Devices and Software section - Full List (177 Items)






1 - Virtual Reality Goes Magnetic with Electronic 'Skin' Sensors
2 - The Enormous Impact of Home Evictions on Mental Health
4 - Slight Fluctuations in Movement Correspond to Autism Diagnoses


