TOPICS > Science

It’s Not Mind-Reading, but Scientists Exploring How Brains Perceive the World

January 2, 2012 at 12:00 AM EDT
It's not mind-reading, but some cutting-edge scientific research could reconstruct brain activity. Jake Schoneker, a recent graduate of the University of California, Berkeley's School of Journalism, and producer Roberto Daza report.


GWEN IFILL: Finally, another in our occasional reports from journalism students around the country.

Tonight, some cutting-edge technology used to tap into the human brain.

The reporter is Jake Schoneker, a recent graduate of the University of California, Berkeley, School of Journalism.

JAKE SCHONEKER: It’s the stuff of science fiction, mind-reading. The power of the human brain has long captured the imaginations of Hollywood.

CHRISTOPHER LLOYD, actor: I’m going to read your thoughts.

MICHAEL J. FOX, actor: Doc.

JAKE SCHONEKER: But as scientists learn more about how the mind works, it could become a reality.

At U.C. Berkeley, Professor Jack Gallant and his team of researchers are in the early stages of piecing together how the mind perceives the world. It turns out it’s a lot like watching a movie.

JACK GALLANT, U.C. Berkeley: You walk around the world and you’re moving your head and your eyes around and you’re looking at scenes, but, in some sense, you’re really just watching a movie of the world going by.

JAKE SCHONEKER: But what if the movies in your mind could be shared with someone else? Gallant’s team did just that. They showed movie trailers to subjects inside this FMRI machine, a scanner designed to measure blood flow in the brain.

JACK GALLANT: You can see there’s some brain activity. This is the brain activity elicited by this movie. And red means more activity and blue means less activity.

JAKE SCHONEKER: They focused on the primary visual cortex, a part of the brain used to process general shapes, textures and movement.

JACK GALLANT: We want to essentially build a dictionary that translates between things that happen in the world and these different parts of your brain and how they respond.

JAKE SCHONEKER: Once they built the dictionary, they used it to comb through thousands of hours of video from YouTube, blending together the frames that matched best with their results. This allowed them to make a rough reconstruction of the movies the subjects saw. The images are far from perfect.

JACK GALLANT: It knows there’s a bright vertical thing with a thing sticking out of it.

JAKE SCHONEKER: But they have been getting attention from scientists and the public alike. They posted their results online and got more than a million views in less than three days.

JACK GALLANT: It showed that people actually do care about science. And they do care about trying to understand the way the brain works. Some people who saw the video thought, wow, this is wonderful. Science is progressing. This is so neat. You can use this for a million things. Some people saw the video and said, oh, my God, this is very scary and frightening and people shouldn’t do this.

JAKE SCHONEKER: But he says critics should know that their work is far from invading your thoughts.

JACK GALLANT: We’re not doing mind-reading here. We’re not really peering into your brain and reconstructing pictures in your head. We’re reading your brain activity and using that brain activity to reconstruct what you saw. And those are two very, very different things.

JAKE SCHONEKER: Also, the equipment needed for FMRI costs millions and weighs several tons, making it unlikely to be used outside of a hospital or research setting.

JACK GALLANT: At some point in the future, people will come up with other methods of measuring brain activity that are more direct and much more accurate.

JAKE SCHONEKER: But not everyone is waiting for next breakthrough. Some companies have adapted a simpler, more stripped-down approach, electroencephalography, or EEG.

Unlike the big expensive equipment need for FMRI, the EEG technology used here at NeuroSky is smaller, cheaper and ready for mainstream use.

DAVID WESTENDORF, NeuroSky: The last ten years, EEG has evolved from a dedicated room with a huge processing box, a statistician that had to go through all of the signal readings, and distilled it into a $99 device.

JAKE SCHONEKER: EEG converts the electrical impulses of the brain into patterns that can be seen on a computer. It’s less precise than FMRI, but it can provide a glimpse of the overall mood of the brain or what are called dominant mental states.

DAVID WESTENDORF: Those can be, are you paying a high level of attention? Are you in a deep state of relaxation?

JAKE SCHONEKER: It turns out that metric, attention and relaxation, provides enough feedback for a wide range of mind-powered applications, mostly in entertainment and games.

TANSY BROOK, NeuroSky: So, we’re actually going to battle our minds against each other.


JAKE SCHONEKER: Here, your level of concentration pushes the ball towards your opponent. When it reaches the other side, you win.

Oh! Vanquished.

COMPUTER VOICE: Player two wins.

TANSY BROOK: This is what we do here.

JAKE SCHONEKER: Some games are designed to help people de-stress and meditate. Others can help children with disorders like ADHD learn to focus their minds.

MAN: “Focus Pocus” is the world’s first ADHD training game that uses brain wave feedback to augment the child’s learning.

TANSY BROOK: So, we don’t really see this as a replacement for the current medical and research technologies, but there’s a lot of more serious applications. So, for example, early diagnosis of Alzheimer’s, going into a doctor’s lab, and just like we check your weight and your blood pressure, now being able to tell if your brain waves look healthy, also for things like PTSD, post-traumatic stress, and traumatic brain injury.

JAKE SCHONEKER: But along with its potential, EEG has its own limitations.

JACK GALLANT: The signal is very, very small. And it can be easily contaminated by things like sweating or blinking. So the quality of the data in EEG is much, much worse than the quality of data in FMRIs.

JAKE SCHONEKER: But this isn’t stopping companies like NeuroSky and others like it from cashing in on these gadgets. Now, whether these products turn out to be more than just toys remains to be seen.

JACK GALLANT: So, there are many companies out there trying to make brain activity recording devices that are portable and cheap and that can be used by just normal people in their daily lives. And I think that’s great, because these cheaper technologies will allow us to apply them to a much wider range of problems.

JAKE SCHONEKER: These advances in neuroscience are setting the stage for new applications that may soon improve our lives. And as these technologies evolve at the juncture between machines and the mind, our idea of science fiction may not be so far-fetched after all.

If only I could do it in real life.

GWEN IFILL: If you’d like to learn more about Professor Gallant’s research, find a link to a conversation about his lab on our website.