Stuart Armstrong
Stuart Armstrong is a philosopher at the University of Oxford and one of the paper's authors.
The Future of Humanity Institute, University of Oxford
Machines are becoming more intelligent every year thanks to advances being made by companies like Google, Facebook, Microsoft, and many others.

AI agents, as they're sometimes known, can already beat us at complex board games like Go, and they're becoming more competent in a range of other areas.

Now a London artificial-intelligence research lab owned by Google has carried out a study to make sure that we can pull the plug on self-learning machines when we want to.

DeepMind, bought by Google for a reported 400 million pounds — about $580 million — in 2014, teamed up with scientists at the University of Oxford to find a way to make sure that AI agents don't learn to prevent, or seek to prevent, humans from taking control.

The paper — "Safely Interruptible Agents PDF," published on the website of the Machine Intelligence Research Institute (MIRI) — was written by Laurent Orseau, a research scientist at Google DeepMind, Stuart Armstrong at Oxford University's Future of Humanity Institute, and several others.

The researchers explain in the paper's abstract that AI agents are "unlikely to behave optimally all the time." They add:


The researchers, who weren't immediately available for interviewing, claim to have created a "framework" that allows a "human operator" to repeatedly and safely interrupt an AI, while making sure that the AI doesn't learn how to prevent or induce the interruptions.

The authors write:


University of Oxford philosopher Nick Bostrom.
The researchers found that some algorithms, such as "Q-learning" ones, are already safely interruptible, while others, like "Sarsa," aren't when they're off the shelf, but they can be modified relatively easily so they are.

"It is unclear if all algorithms can be easily made safely interruptible," the authors admit.

DeepMind's work with the Future of Humanity Institute is interesting: DeepMind wants to "solve intelligence" and create general purpose AIs, while the Future of Humanity Institute is researching potential threats to our existence. The institute is led by Nick Bostrom, who believes that machines will outsmart humans within the next 100 years and thinks that they have the potential to turn against us.

Speaking at Oxford University in May 2015 at the annual Silicon Valley Comes to Oxford event, Bostrom said:

I personally believe that once human equivalence is reached, it will not be long before machines become superintelligent after that. It might take a long time to get to human level but I think the step from there to superintelligence might be very quick. I think these machines with superintelligence might be extremely powerful, for the same basic reasons that we humans are very powerful relative to other animals on this planet. It's not because our muscles are stronger or our teeth are sharper, it's because our brains are better.

DeepMind knows the technology that it's creating has the potential to cause harm. The founders — Demis Hassabis, Mustafa Suleyman, and Shane Legg — allowed their company to be bought by Google on the condition that the search giant created an AI ethics board to monitor advances that Google makes in the field. Who sits on this board and what they do, exactly, remains a mystery.

The founders have also attended and spoken at several conferences about ethics in AI, highlighting that they want to ensure the technology they and others are developing is used for good, not evil. It's likely that they will look to incorporate some of the findings from the "Safely Interruptible Agents" paper into their work going forward.

Get the latest Google stock price here.