Scientists Decode Words From Brain Signals

September 7, 2010




The Journal of Neural Engineering’s September issue is publishing Greger’s study showing the feasibility of translating brain signals into computer-spoken words.

The University of Utah research team placed grids of tiny microelectrodes over speech centers in the brain of a volunteer with severe epileptic seizures. The man already had a craniotomy ““ temporary partial skull removal ““ so doctors could place larger, conventional electrodes to locate the source of his seizures and surgically stop them.


Later, they tried figuring out which brain signals represented each of the 10 words. When they compared any two brain signals ““ such as those generated when the man said the words “yes” and “no” ““ they were able to distinguish brain signals for each word 76 percent to 90 percent of the time.

When they examined all 10 brain signal patterns at once, they were able to pick out the correct word any one signal represented only 28 percent to 48 percent of the time ““ better than chance (which would have been 10 percent) but not good enough for a device to translate a paralyzed person’s thoughts into words spoken by a computer.


People who eventually could benefit from a wireless device that converts thoughts into computer-spoken spoken words include those paralyzed by stroke, Lou Gehrig’s disease and trauma, Greger says. People who are now “locked in” often communicate with any movement they can make ““ blinking an eye or moving a hand slightly ““ to arduously pick letters or words from a list.



Nonpenetrating Microelectrodes Read Brain’s Speech Signals




Because the microelectrodes do not penetrate brain matter, they are considered safe to place on speech areas of the brain ““ something that cannot be done with penetrating electrodes that have been used in experimental devices to help paralyzed people control a computer cursor or an artificial arm.


Translating Nerve Signals into Words


Each of two grids with 16 microECoGs spaced 1 millimeter (about one-25th of an inch) apart, was placed over one of two speech areas of the brain: First, the facial motor cortex, which controls movements of the mouth, lips, tongue and face ““ basically the muscles involved in speaking. Second, Wernicke’s area, a little understood part of the human brain tied to language comprehension and understanding.

The study was conducted during one-hour sessions on four consecutive days. Researchers told the epilepsy patient to repeat one of the 10 words each time they pointed at the patient. Brain signals were recorded via the two grids of microelectrodes. Each of the 10 words was repeated from 31 to 96 times, depending on how tired the patient was. Then the researchers “looked for patterns in the brain signals that correspond to the different words” by analyzing changes in strength of different frequencies within each nerve signal, says Greger.


One unexpected finding: When the patient repeated words, the facial motor cortex was most active and Wernicke’s area was less active. Yet Wernicke’s area “lit up” when the patient was thanked by researchers after repeating words. It shows Wernicke’s area is more involved in high-level understanding of language, while the facial motor cortex controls facial muscles that help produce sounds, Greger says.

The researchers were most accurate ““ 85 percent ““ in distinguishing brain signals for one word from those for another when they used signals recorded from the facial motor cortex. They were less accurate ““ 76 percent ““ when using signals from Wernicke’s area. Combining data from both areas didn’t improve accuracy, showing that brain signals from Wernicke’s area don’t add much to those from the facial motor cortex.


In the more difficult test of distinguishing brain signals for one word from signals for the other nine words, the researchers initially were accurate 28 percent of the time ““ not good, but better than the 10 percent random chance of accuracy. However, when they focused on signals from the five most accurate electrodes, they identified the correct word almost half (48 percent) of the time.

“It doesn’t mean the problem is completely solved and we can all go home,” Greger says. “It means it works, and we now need to refine it so that people with locked-in syndrome could really communicate.”

“The obvious next step ““ and this is what we are doing right now ““ is to do it with bigger microelectrode grids” with 121 micro electrodes in an 11-by-11 grid, he says. “We can make the grid bigger, have more electrodes and get a tremendous amount of data out of the brain, which probably means more words and better accuracy.”

Image 1: An array of 16 microelectrodes — known as a microECoG grid — is arranged in a four-by-four array and shown next to a US quarter-dollar coin with a Utah state design on its “tail” side. University of Utah researchers placed two such microelectrode grids over speech areas of a patient’s brain and used them to decode brain signals into words. The technology someday might help severely paralyzed patients “speak” with their thoughts, which would be converted into a computerized voice. Credit: Spencer Kellis, University of Utah

Image 2: This photo shows two kinds of electrodes sitting atop a severely epileptic patient’s brain after part of his skull was removed temporarily. The larger, numbered, button-like electrodes are ECoGs used by surgeons to locate and then remove brain areas responsible for severe epileptic seizures. While the patient had to undergo that procedure, he volunteered to let researchers place two small grids — each with 16 tiny “microECoG” electrodes — over two brain areas responsible for speech. These grids are at the end of the green and orange wire bundles, and the grids are represented by two sets of 16 white dots since the actual grids cannot be seen easily in the photo. University of Utah scientists used the microelectrodes to translate speech-related brain signals into actual words — a step toward future machines to allow severely paralyzed people to speak. Credit: University of Utah Department of Neurosurgery

Image 3: This magnetic resonance image (MRI) of an epileptic patient’s brain is superimposed with the locations of two kinds of electrodes: conventional ECoG electrodes (yellow) to help locate the source of his seizures so surgeons could operate to prevent them, and two grids (red) of 16 experimental microECoG electrodes used to read speech signals from the brain. University of Utah scientists used the microelectrodes to translate brain signals into words — a step toward devices that would let severely paralyzed people speak. Credit: Kai Miller, University of Washington

On the Net:

Scientists Decode Words From Brain Signals Scientists Decode Words From Brain Signals

comments powered by Disqus