28 March 2012

Kinect lab boss on the future of computer interfaces

It's just the beginning

Andrew Blake is managing director of Microsoft Research Cambridge, the UK lab responsible for the machine learning software behind Kinect. New Scientist caught up with him at The Economist’s Technology Frontiers conference last week to discuss his vision for the future of computer interfaces – and how, one day, we might all think like Meryl Streep.

How have computer interfaces evolved over time?
We’ve been through three generations of how we interact with computers. Green screen and cursor and keyboard was first, then we had mouse and Windows, followed by touch and multi-touch, and now it looks like we may be getting a fourth generation which is no touch, or action at a distance. Kinect is a piece of this, but there’s really a lot of technology coming up from other labs.

So what’s next for Kinect? It has made it into millions of gamers’ homes and is constantly being tinkered with by hobbyists and researchers, but will it ever have more serious uses?
It’s only in millions of homes – I’d love to see it in hundreds of millions of homes. I do think the real potential of the technology is how it changes the way we think about interfacing with machines of all kinds, and I think we’re absolutely at the beginning of that. It’s as if we are at the stage where somebody has just found out what metal film you could use to make a touchscreen, and now some creative people have got touchscreens and they’re beginning to play with them.

Will we eventually all be like Tom Cruise in Minority Report, waving our arms around at our desk?
I don’t know if we ever knew what Tom Cruise was really doing, he was waving his hands and videos were moving around and they were trying to sell you the line that he was sort of composing pieces of information in this very intelligent way, but I’m not sure that’s really what was happening.


I find that vision somehow not quite right, but something not very far from it would be giving a presentation. I probably would like a good PowerPoint user interface where you’re almost talking or signalling to the projector and it’s your “apprentice” – I think that notion of an apprentice is important.

Newer interface generations won’t replace the mouse and keyboard then?
There’s no reason why they should, the number of modalities that we have to play with now is expanding. Something that humans seem to be very good at is using parallel channels, but have we really exploited that with a computer? I think Kinect is bringing in a very important extra parallel channel but it’s not the only one.

People will still be at their workstations with keyboard and mouse, the mouse is a wonderful precision instrument when you have to do that kind of work, but it’s not so natural for other things. If you don’t want to be concentrating quite so closely you may like to be more approximate, and I think that’s what people love about the modern, multi-touch paradigm. It’s sort of more relaxed and when you’re in that kind of mood that’s a nice way to interact.

Using exotic interfaces such as gesture or voice commands can be a bit embarrassing though, do social norms need to change for these interfaces to really take off?
We used to think that people who wandered in the street, with earphones in their ears, talking to nobody were nutters. Now we realise they’re just on the phone. It doesn’t seem so odd anymore. So, I think things that give you enough leverage stand a chance of becoming social norms.

And what’s next, once those become the norm?
If I knew what the fifth generation was I wouldn’t be sitting here! Brain-computer interfaces are amazing, let’s see how well they can be made to work. For the best ones, of course, you need an implant, but if non-contact brain-computer interfaces get better that would be marvellous.

What would be the killer app of a thought-controlled computer?
Thought control might be at its best when we are wandering around the street. As we said, it’s already acceptable to have ear phones plugged into your head, but if I want to find my way through the city I’m going to look a bit of wally chatting about where I want to go. If I could replace that by thinking, great.

Even more so, there’s what I think of as a The Devil Wears Prada scenario. Meryl Streep is at a party and wants to network but she can’t actually remember everybody’s name, so her two lovely assistants have memorised an entire album full of faces and biographies. Whenever someone comes up they whisper to her who it is, so she goes “Oh, darling, John!” as if she knew who it was. If I could have that in my phone, it would be fantastic.