(skip this header)

Sunday, April 26, 2015

Times Union
timesunion.com Businesses

« Back to Article

RPI researchers work to give robots the knowledge of good and evil

An RPI research team seeks a way to instill in machines a sense of human morality
Brian Nearin, Times Union
Published 8:30 pm, Saturday, March 1, 2014


  • Selmer Bringsjord, Department of Cognitive Science chair, talks about the future of robotics on Tuesday, Feb. 25, 2014, at Rensselaer Polytechnic Institute in Troy, N.Y. Bringsjord is part of a research project, funded by the U.S. military, on how to bring ethics into the world of robotics. (Cindy Schultz / Times Union) Photo: Cindy Schultz / 00025868A


Larger | Smaller
Email This
Page 1 of 1


Showing children, and even some adults, the difference between good and bad can be a frustrating experience.

So how might complex and sometimes contradictory ideas of human morality be imparted into an artificially intelligent machine?

At Rensselaer Polytechnic Institute, 55-year-old Selmer Bringsjord is working on this problem, which once existed only in the minds of science-fiction writers. Now it is poised to enter the real world as robots — from unmanned aerial drones to self-driving vehicles — are developed and brought into use by the military and industry.

Because robots are expected to perform increasingly outside continuous oversight and control of humans, somehow machines must be made to "understand" not to harm their masters — or, more simply, not to do evil. Morality is considered the expression of the highest human character, and is interwoven with ideas of religion and empathy.

Whether robots would be amoral and thus capable of turning on humanity is a theme that has captured the human imagination since the idea of mechanical robots surfaced in the 1920s. This fear has since inspired movies like "Terminator," "Blade Runner" and the German silent film classic "Metropolis."

In his offices, Bringsjord asked a visitor to imagine how a robot delivery vehicle might approach the morality of finding an injured person on its route. It had been programmed to deliver its payload without delay, which might be critical for a military mission. Stopping to assist the injured human might jeopardize this mission. But not helping the person could also cause harm. What to do? What if the human is a known ally? A potential adversary? And how to express all the ramifications behind such a choice in computer code? And how to do it so the robot can react quickly, rather than waste precious time while it runs through infinite scenarios?

Closer to civilian life, what about a robot-controlled car, a concept that Google has already developed? The robot might be programmed never to let an intoxicated person operate the vehicle to reduce the risk of an accident that could harm others. But what if such a driver needed to rush a gravely ill child to the hospital?

"The idea of not potentially harming a human also would not be feasible for a robotic surgeon, for example," said Bringsjord, who heads RPI's Department of Cognitive Science and its Artificial Intelligence and Reasoning Lab.

His research relies on the development of increasing levels of artificial intelligence, or AI, in computer systems, and then using that computing power to contain and employ concepts of morality, expressed as algorithms in programming language. What humans can choose through free will, and have developed through experience, philosophy and religious strictures, machines will have to grasp through mathematics and logic.

Robotic vacuum cleaners and lawn mowers are already here. Anyone with an Apple iPhone with the Siri feature is already being exposed to "weak AI," Bringsjord said.

But developers are working on more intelligent robots, like those envisioned to assist in the care of elderly people. So stronger AI is coming, and machines using it will need an understanding of humans and the fact that humans are mortal, in order to understand morality, Bringsjord said.

"The machines will have to understand us, in order to dialogue with us, and navigate the terrain with us," he said. "They will have to have an understanding of our mortality, and of its own mortality, that it is aware that it can be decommissioned."

Bringsjord will soon be expanding his robot-ethics research program courtesy of a recent multimillion-dollar, multi-university award from the Office of Naval Research, part of which is to support his RPI team.

With help from RPI colleague Mei Si, assistant professor in the Department of Cognitive Science and an expert in the computational modeling of emotions, and post-doctoral researcher Naveen Govindarajulu, the team aims to capture emotions in programming language.

In May, Bringsjord will demonstrate his work in Chicago, where two robots will be put through their paces. One is programmed to succumb to the temptation of revenge, and another — controlled by the moral logic being engineered — is to resist vengeance and violence.

The Office of Naval Research has been interested in so-called moral machines for several years now. According to a 2008 research paper written for the office by researchers at California Polytechnic State University, there are ways to impart concepts of right and wrong to so-called "artificial moral agents."

Military planners want robots that are "safe, and do no harm to (friendly) military personnel, civilians and other agents worthy of moral consideration," the report states. One approach is called top-down, which is done by programming in rules, such as do no harm. Imagine programing into a robot's operating system teachings from the Bible, Plato's Republic, or the Dali Lama.

"Personally, I don't see how you can separate the math and the logic of machine morality from the religious dimension. But that is not part of this project," Bringsjord said.

But the top-down might still lead to terrible consequences, if the robot is focused only on obeying rules, rather than weighing outcomes, especially if outcomes were unforeseen by engineers and thus unaddressed in the robot's programming.

Another approach is called by bottom-up, in which machines are envisioned somewhat like children, and are given less precise rules, but then essentially "learn" by trial and error to build a comprehensive, effective moral framework. But this presents challenges, too, if the robot encounters a novel situation beyond its experience.

Bringsjord believes that morality will have to be programmed into a robot's operating system, or "in the bloodstream," so it cannot be disabled or bypassed. This would make it difficult for captured military robots to be used against their former allies, and would render impossible the idea of a robotic turncoat.

And it would also make robots incapable of committing wartime atrocities. "If the programming is correct, the idea that a robot could invoke the Nuremberg Defense (used by Nazis after World War II that they were only following orders) would not be possible," he said.

For good to exist, there must be evil. And Bringsjord has already explored that in a digital realm. In 2008, his lab created a software program for a malevolent entity called "E," which was meant to display only wicked traits in response to questions. But no need to worry about encountering "E" in any online gaming room; Bringsjord said that work is currently dormant and E exists only on a hard drive in the lab.

"It is always easier to do the negative than the positive," he said. "There is lot more previous work on this, like abnormal psychology. There is no field called psycho-virtuosity."


Featured videos