0
SPEAKER_01
Welcome to the Artificial Intelligence Podcast.
0
SPEAKER_01
My name is Lex Friedman.
0
SPEAKER_01
I m a research scientist at MIT.
0
SPEAKER_01
Today is a conversation with Steven Pinker.
0
SPEAKER_01
He s a professor at Harvard and before that was a professor at MIT.
0
SPEAKER_01
He s the author of many books, some of which had a big impact on the way I see the world for the better.
0
SPEAKER_01
In particular, The Better Angels of Our Nature and his latest book, Enlightenment Now, have instilled in me a sense of optimism.
0
SPEAKER_01
Optimism grounded in data, science, and reason.
0
SPEAKER_01
I really enjoyed this conversation.
0
SPEAKER_01
I hope you do as well.
0
SPEAKER_02
You ve studied the human mind, cognition, language, vision, evolution, psychology, from child to adult, from the level of individual to the level of our entire civilization, so I feel like I can start with a simple multiple choice question.
0
SPEAKER_02
What is the meaning of life?
0
SPEAKER_02
Is it A, to attain knowledge, as Plato said?
0
SPEAKER_02
B, to attain power, as Nietzsche said?
0
SPEAKER_02
C, to escape death, as Ernest Becker said?
0
SPEAKER_02
D, to propagate our genes, as Darwin and others have said?
0
SPEAKER_02
E, there is no meaning, as the nihilists have said?
0
SPEAKER_02
F, knowing the meaning of life is beyond our cognitive capabilities, as Stephen Pinker said, based on my interpretation 20 years ago?
0
SPEAKER_02
and G, none of the above.
0
SPEAKER_00
I d say aid comes closest, but I would amend that to attaining not only knowledge, but fulfillment more generally.
0
SPEAKER_00
That is life, health, stimulation, access to the living cultural and social world.
0
SPEAKER_00
Now this is our meaning of life.
0
SPEAKER_00
It s not the meaning of life if you were to ask our genes.
0
SPEAKER_00
Their meaning is to propagate copies of themselves.
0
SPEAKER_00
But that is distinct from the meaning that the brain that they lead to sets for itself.
0
SPEAKER_02
So to you, knowledge is a small subset or a large subset?
0
SPEAKER_00
It s a large subset, but it s not the entirety of human striving.
0
SPEAKER_00
Because we also want to interact with people, we want to experience beauty, we want to experience the richness of the natural world.
0
SPEAKER_00
But understanding what makes the universe tick is way up there.
0
SPEAKER_00
For some of us more than others, certainly for me that s one of the top five.
0
SPEAKER_02
So is that a fundamental aspect?
0
SPEAKER_02
Are you just describing your own preference or is this a fundamental aspect of human nature is to seek knowledge?
0
SPEAKER_02
In your latest book you talk about the power, the usefulness of rationality and reason and so on.
0
SPEAKER_02
Is that a fundamental nature of human beings or is it something we should just strive for?
0
SPEAKER_00
It s both.
0
SPEAKER_00
We re capable of striving for it because it is one of the things that make us what we are.
0
SPEAKER_00
Homo sapiens, wise men.
0
SPEAKER_00
We are unusual among animals in the degree to which we acquire knowledge and use it to survive.
0
SPEAKER_00
We make tools, we strike agreements via language, we extract poisons, we predict the behavior of animals, we try to get at the workings of plants.
0
SPEAKER_00
And when I say we, I don t just mean we in the modern West, but we as a species everywhere, which is how we ve managed to
0
SPEAKER_00
occupy every niche on the planet, how we ve managed to drive other animals to extinction, and the refinement of reason in pursuit of human well-being, of health, happiness, social richness, cultural richness.
0
SPEAKER_00
is our main challenge in the present.
0
SPEAKER_00
That is, using our intellect, using our knowledge to figure out how the world works, how we work, in order to make discoveries and strike agreements that make us all better off in the long run.
0
SPEAKER_02
And you do that almost undeniably and in a data-driven way in your recent book.
0
SPEAKER_02
But I d like to focus on the artificial intelligence aspect of things, and not just artificial intelligence, but natural intelligence too.
0
SPEAKER_02
So 20 years ago in the book you ve written on how the mind works, you conjecture, again, am I right to interpret things?
0
SPEAKER_02
You can correct me if I m wrong, but you conjecture that human thought in the brain may be a result of
0
SPEAKER_02
a massive network of highly interconnected neurons, so from this interconnectivity emerges thought.
0
SPEAKER_02
Compared to artificial neural networks which we use for machine learning today, is there something fundamentally more complex, mysterious, even magical about the biological neural networks versus the ones we have been starting to use over the past 60 years and have become to success in the past 10?
0
SPEAKER_00
There is something a little bit mysterious about the human neural networks, which is that each one of us who is a neural network knows that we ourselves are conscious.
0
SPEAKER_00
Conscious not in the sense of registering our surroundings or even registering our internal state.
0
SPEAKER_00
but in having subjective, first-person, present-tense experience.
0
SPEAKER_00
That is, when I see red, it s not just different from green, but there s a redness to it that I feel.
0
SPEAKER_00
Whether an artificial system would experience that or not, I don t know, and I don t think I can know.
0
SPEAKER_00
That s why it s mysterious.
0
SPEAKER_00
If we had a perfectly lifelike robot that was behaviorally indistinguishable from a human, would we attribute consciousness to it, or ought we to attribute consciousness to it?
0
SPEAKER_00
and that s something that it s very hard to know.
0
SPEAKER_00
But putting that aside, putting aside that largely philosophical question, the question is, is there some difference between the human neural network and the ones that we re building in artificial intelligence will mean that we re, on the current trajectory, not gonna reach the point where we ve got a lifelike robot indistinguishable from a human because the way their so-called neural networks are organized are different from the way ours are organized.
0
SPEAKER_00
I think there s overlap, but I think there are some big differences.
0
SPEAKER_00
The current neural networks, current so-called deep learning systems, are in reality not all that deep.
0
SPEAKER_00
That is, they are very good at extracting high-order statistical regularities.
0
SPEAKER_00
But most of the systems don t have a semantic level, a level of actual understanding of who did what to whom, why, where, how things work, what causes what else.
0
SPEAKER_02
Do you think that kind of thing can emerge as it does, so artificial neural networks are much smaller, the number of connections and so on, than the current human biological networks, but do you think, sort of, to go to consciousness or to go to this higher level semantic reasoning about things, do you think that can emerge with just a larger network, with a more richly, weirdly interconnected network?
0
SPEAKER_00
Separate, again, consciousness, because consciousness isn t even a matter of complexity.
0
SPEAKER_00
You could sensibly ask the question of whether shrimp are conscious, for example.
0
SPEAKER_00
They re not terribly complex, but maybe they feel pain.
0
SPEAKER_00
So let s just put that part of it aside.
0
SPEAKER_00
I think sheer size of a neural network is not enough to give it structure and knowledge.
0
SPEAKER_00
But if it s suitably engineered, then why not?
0
SPEAKER_00
That is, we re neural networks.
0
SPEAKER_00
Natural selection did a kind of equivalent of engineering of our brains.
0
SPEAKER_00
So I don t think there s anything mysterious in the sense that no system made out of silicon could ever do what a human brain can do.
0
SPEAKER_00
I think it s possible in principle.
0
SPEAKER_00
Whether it ll ever happen depends not only on how clever we are in engineering these systems, but whether we even want to, whether that s even a sensible goal.
0
SPEAKER_00
That is, you can ask the question, is there any locomotion system that is as good as a human?
0
SPEAKER_00
Well, we kind of want to do better than a human, ultimately, in terms of legged locomotion.
0
SPEAKER_00
There s no reason that humans should be our benchmark.
0
SPEAKER_00
They re tools that might be better in some ways.
0
SPEAKER_00
It may be that we can t duplicate a natural system because at some point it s so much cheaper to use a natural system that we re not going to invest more brain power and resources.
0
SPEAKER_00
So for example, we don t really have an exact substitute for wood.
0
SPEAKER_00
We still build houses out of wood.
0
SPEAKER_00
We still build furniture out of wood.
0
SPEAKER_00
We like the look.
0
SPEAKER_00
We like the feel.
0
SPEAKER_00
Wood has certain properties that synthetics don t.
0
SPEAKER_00
It s not that there s anything magical or mysterious about wood.
0
SPEAKER_00
It s just that the extra steps of duplicating everything about wood is something we just haven t bothered because we have wood.
0
SPEAKER_00
Like when I say cotton.
0
SPEAKER_00
I mean, I m wearing cotton clothing now.
0
SPEAKER_00
It feels much better than polyester.
0
SPEAKER_00
It s not that cotton has something magic in it, and it s not that we couldn t ever synthesize something exactly like cotton, but at some point, it s just not worth it.
0
SPEAKER_00
We ve got cotton.
0
SPEAKER_00
And likewise, in the case of human intelligence, the goal of making an artificial system that is exactly like the human brain
0
SPEAKER_00
is a goal that we probably no one is gonna pursue to the bitter end, I suspect.
0
SPEAKER_00
Because if you want tools that do things better than humans, you re not gonna care whether it does something like humans.
0
SPEAKER_00
So for example, diagnosing cancer or predicting the weather, why set humans as your benchmark?
0
SPEAKER_02
But in general, I suspect you also believe that even if the human should not be a benchmark, and we don t want to imitate humans in their system, there s a lot to be learned about how to create an artificial intelligence system by studying the human.
0
SPEAKER_00
Yeah, I think that s right.
0
SPEAKER_00
In the same way that to build flying machines, we want to understand the laws of aerodynamics, including birds, but not mimic the birds.
0
SPEAKER_00
But they re the same laws.
0
SPEAKER_02
You have a view on AI, artificial intelligence and safety, that from my perspective is refreshingly rational, or perhaps more importantly, has elements of positivity to it, which I think can be inspiring and empowering as opposed to paralyzing.
0
SPEAKER_02
For many people, including AI researchers, the eventual existential threat of AI is obvious.
0
SPEAKER_02
Not only possible, but obvious.
0
SPEAKER_02
And for many others, including AI researchers, the threat is not obvious.
0
SPEAKER_02
So, Elon Musk is famously in the highly concerned about AI camp, saying things like AI is far more dangerous than nuclear weapons, and that AI will likely destroy human civilization.
0
SPEAKER_02
So in February you said that if Elon was really serious about AI, the threat of AI, he would stop building self-driving cars that he s doing very successfully as part of Tesla.
0
SPEAKER_02
Then he said, wow, if even Pinker doesn t understand the difference between narrow AI like a car and general AI,
0
SPEAKER_02
When the latter literally has a million times more compute power and an open-ended utility function, humanity is in deep trouble.
0
SPEAKER_02
So first, what did you mean by the statement about Elon Musk should stop billing self-driving cars if he s deeply concerned?
0
SPEAKER_00
Not the last time that Elon Musk has fired off an intemperate tweet.
0
SPEAKER_02
Well, we live in a world where Twitter has power.
0
SPEAKER_02
Yes.
0
SPEAKER_00
Yeah, I think that there are two kinds of existential threat that have been discussed in connection with artificial intelligence, and I think that they re both incoherent.
0
SPEAKER_00
One of them is a vague fear of AI takeover.
0
SPEAKER_00
It just as we subjugated animals and less technologically advanced peoples, so if we build something that s more advanced than us, it will inevitably turn us into pets or slaves or domesticated animal equivalents.
0
SPEAKER_00
I think this confuses intelligence with a will to power.
0
SPEAKER_00
It so happens that in the intelligence system we are most familiar with, namely Homo sapiens, we are products of natural selection, which is a competitive process, and so bundled together with our problem-solving capacity are a number of nasty traits like dominance and exploitation and maximization of power and glory and resources
0
SPEAKER_00
and influence, there s no reason to think that sheer problem-solving capability will set that as one of its goals.
0
SPEAKER_00
Its goals will be whatever we set its goals as, and as long as someone isn t building a megalomaniacal artificial intelligence.
0
SPEAKER_00
And there s no reason to think that it would naturally evolve in that direction.
0
SPEAKER_00
Now, you might say, well, what if we gave it the goal of maximizing its own power source?
0
SPEAKER_00
That s a pretty stupid goal to give an autonomous system.
0
SPEAKER_00
You don t give it that goal.
0
SPEAKER_00
I mean, that s just self-evidently idiotic.
0
SPEAKER_02
So if you look at the history of the world, there s been a lot of opportunities where engineers could instill in a system destructive power, and they choose not to, because that s the natural process of engineering.
0
SPEAKER_00
Well, except for weapons.
0
SPEAKER_00
I mean, if you re building a weapon, its goal is to destroy people.
0
SPEAKER_00
And so I think there are good reasons to not build certain kinds of weapons.
0
SPEAKER_00
I think building nuclear weapons was a massive mistake.
0
SPEAKER_02
You do.
0
SPEAKER_02
So maybe pause on that because that is one of the serious threats.
0
SPEAKER_02
Do you think that it was a mistake in a sense that it should have been stopped early on?
0
SPEAKER_02
Or do you think it s just an unfortunate event of invention that this was invented?
0
SPEAKER_02
Do you think it was possible to stop, I guess, is the question I m asking.
0
SPEAKER_00
It s hard to rewind the clock because, of course, it was invented in the context of World War II and the fear that the Nazis might develop one first.
0
SPEAKER_00
Then once it was initiated for that reason, it was hard to turn off, especially since winning the war against the Japanese and the Nazis was such an overwhelming goal of every responsible person that there was just nothing that people wouldn t have done then to ensure victory.
0
SPEAKER_00
It s quite possible, if World War II hadn t happened, that nuclear weapons wouldn t have been invented.
0
SPEAKER_00
We can t know.
0
SPEAKER_00
But I don t think it was by any means a necessity, any more than some of the other weapon systems that were envisioned but never implemented, like planes that would disperse poison gas over cities, like crop dusters, or systems to try to create earthquakes and tsunamis in enemy countries.
0
SPEAKER_00
to weaponize the weather, weaponize solar flares, all kinds of crazy schemes that we thought the better of.
0
SPEAKER_00
I think analogies between nuclear weapons and artificial intelligence are fundamentally misguided, because the whole point of nuclear weapons is to destroy things.
0
SPEAKER_00
The point of artificial intelligence is not to destroy things.
0
SPEAKER_00
So the analogy is misleading.
0
SPEAKER_02
So there s two artificial intelligence you mentioned, the first one that gets highly intelligent or power-hungry.
0
SPEAKER_00
Yeah, it s a system that we design ourselves, where we give it the goals.
0
SPEAKER_00
Goals are external to the means to attain the goals.
0
SPEAKER_00
If we don t design an artificially intelligent system to maximize dominance, then it won t maximize dominance.
0
SPEAKER_00
It s just that we re so familiar with Homo sapiens, where these two traits come bundled together, particularly in men, that we are apt to confuse high intelligence with a will to power.
0
SPEAKER_00
But that s just an error.
0
SPEAKER_00
The other fear is that we ll be collateral damage, that we ll give artificial intelligence a goal, like make paperclips, and it will pursue that goal so brilliantly that before we can stop it, it turns us into paperclips.
0
SPEAKER_00
We ll give it the goal of curing cancer, and it will turn us into guinea pigs for lethal experiments.
0
SPEAKER_00
Or we ll give it the goal of world peace, and its conception of world peace is no people, therefore no fighting, and so it will kill us all.
0
SPEAKER_00
Now, I think these are utterly fanciful.
0
SPEAKER_00
In fact, I think they re actually self-defeating.
0
SPEAKER_00
They, first of all, assume that we re going to be so brilliant that we can design an artificial intelligence that can cure cancer, but so stupid that we don t specify what we mean by curing cancer in enough detail that it won t kill us in the process.
0
SPEAKER_00
And it assumes that the system will be so smart that it can cure cancer, but so idiotic that it can t figure out that what we mean by curing cancer is not killing everyone.
0
SPEAKER_00
So, I think that the collateral damage scenario, the value alignment problem, is also based on a misconception.
0
SPEAKER_02
So, one of the challenges, of course, we don t know how to build either system currently, or are we even close to knowing?
0
SPEAKER_02
Of course, those things can change overnight, but at this time, theorizing about it is very challenging in either direction.
0
SPEAKER_02
So, that s probably at the core of the problem.
0
SPEAKER_02
without that ability to reason about the real engineering things here at hand is your imagination runs away with things.
0
SPEAKER_00
Exactly.
0
SPEAKER_02
But let me sort of ask, what do you think was the motivation and the thought process of Elon Musk?
0
SPEAKER_02
I build autonomous vehicles.
0
SPEAKER_02
I study autonomous vehicles.
0
SPEAKER_02
I studied Tesla Autopilot.
0
SPEAKER_02
I think it is one of the greatest currently large-scale applications of artificial intelligence in the world.
0
SPEAKER_02
It has potentially a very positive impact on society.
0
SPEAKER_02
So how does a person who s creating this very good, quote-unquote, narrow AI system, also seem to be so concerned about this other general AI?
0
SPEAKER_02
What do you think is the motivation there?
0
SPEAKER_02
What do you think is the problem?
0
SPEAKER_00
Well, you probably have to ask him.
0
SPEAKER_00
But there, and he is notoriously flamboyant, impulsive, to the, as we have just seen, to the detriment of his own goals of the health of a company.
0
SPEAKER_00
So I don t know what s going on in his mind.
0
SPEAKER_00
You probably have to ask him.
0
SPEAKER_00
But I don t think the, and I don t think the distinction between special purpose AI and so-called general AI is relevant.
0
SPEAKER_00
that in the same way that special purpose AI is not going to do anything conceivable in order to attain a goal, all engineering systems are designed to trade off across multiple goals.
1
SPEAKER_00
build cars in the first place, we didn t forget to install brakes, because the goal of a car is to go fast.
1
SPEAKER_00
It occurred to people, yes, you want it to go fast, but not always, so you build in brakes too.
1
SPEAKER_00
Likewise, if a car is going to be autonomous, and program it to take the shortest route to the airport,
1
SPEAKER_00
It s not going to take the diagonal and mow down people and trees and fences because that s the shortest route.
1
SPEAKER_00
That s not what we mean by the shortest route when we program it, and that s just what an intelligent system is by definition.
1
SPEAKER_00
It takes into account multiple constraints.
1
SPEAKER_00
The same is true, in fact, even more true of so-called general intelligence.
1
SPEAKER_00
That is, if it s genuinely intelligent, it s not going to pursue some goal single-mindedly, omitting every other
1
SPEAKER_00
consideration and collateral effect.
1
SPEAKER_00
That s not artificial and general intelligence.
1
SPEAKER_00
That s artificial stupidity.
1
SPEAKER_00
I agree with you, by the way, on the promise of autonomous vehicles for improving human welfare.
1
SPEAKER_00
I think it s spectacular.
1
SPEAKER_00
And I m surprised at how little press coverage notes that in the United States alone, something like 40,000 people die every year on the highways, vastly more than are killed by terrorists.
1
SPEAKER_00
And we spent a trillion dollars on a war to combat
1
SPEAKER_00
deaths by terrorism, about half a dozen a year, whereas year in, year out, 40,000 people are massacred on the highways, which could be brought down to very close to zero.
1
SPEAKER_00
So I m with you on the humanitarian benefit.
1
SPEAKER_01
Let me just mention that it s, as a person who s building these cars, it is a little bit offensive to me to say that engineers would be clueless enough not to engineer safety into systems.
1
SPEAKER_01
I often stay up at night thinking about those 40,000 people that are dying.
1
SPEAKER_01
And everything I try to engineer is to save those people s lives.
1
SPEAKER_01
So every new invention that I m super excited about, every new
1
SPEAKER_01
and in all the deep learning literature and CVPR conferences and NIPS, everything I m super excited about is all grounded in making it safe and help people.
1
SPEAKER_01
So I just don t see how that trajectory can all of a sudden slip into a situation where intelligence will be highly negative.
1
SPEAKER_00
I just want to- You and I certainly agree on that, and I think that s only the beginning of the potential humanitarian benefits of artificial intelligence.
1
SPEAKER_00
There s been enormous attention to what are we gonna do with the people whose jobs are made obsolete by artificial intelligence, but very little attention given to the fact that the jobs that are gonna be made obsolete are horrible jobs.
1
SPEAKER_00
The fact that people aren t gonna be picking crops and making beds and driving trucks and mining coal, these are soul-deadening jobs.
1
SPEAKER_00
And we have a whole literature sympathizing with the people stuck in these menial,
1
SPEAKER_00
mind-deadening, dangerous jobs.
1
SPEAKER_00
If we can eliminate them, this is a fantastic boon to humanity.
1
SPEAKER_00
Now, granted, you solve one problem and there s another one, namely, how do we get these people a decent income?
1
SPEAKER_00
But if we re smart enough to invent machines that can make beds and put away dishes and handle hospital patients,
1
SPEAKER_00
I think we re smart enough to figure out how to redistribute income to apportion some of the vast economic savings to the human beings who will no longer be needed to make beds.
1
SPEAKER_01
Okay, Sam Harris says that it s obvious that eventually AI will be an existential risk.
1
SPEAKER_01
He s one of the people who says it s obvious.
1
SPEAKER_01
We don t know when the claim goes, but eventually it s obvious, and because we don t know when, we should worry about it now.
1
SPEAKER_01
So it s a very interesting argument in my eyes.
1
SPEAKER_01
So how do we think about,
1
SPEAKER_01
timescale, how do we think about existential threats when we don t really, we know so little about the threat, unlike nuclear weapons perhaps, about this particular threat, that it could happen tomorrow, right?
1
SPEAKER_01
So, but very likely it won t.
1
SPEAKER_01
Very likely it d be a hundred years away.
1
SPEAKER_01
So how do we ignore it?
1
SPEAKER_01
How do we talk about it?
1
SPEAKER_01
Do we worry about it?
1
SPEAKER_01
How do we think about those?
1
SPEAKER_00
What is it?
1
SPEAKER_01
A threat that we can imagine, it s within the limits of our imagination, but not within our limits of understanding to accurately predict it.
1
SPEAKER_00
But what is the it that we re afraid of?
1
SPEAKER_01
Oh, AI, sorry, AI being the existential threat.
1
SPEAKER_00
But how, like enslaving us or turning us into paperclips?
1
SPEAKER_01
I think the most compelling from the Sam Harris perspective would be the paperclip situation.
1
SPEAKER_00
Yeah, I just think it s totally fanciful.
1
SPEAKER_00
Don t build a system.
1
SPEAKER_00
First of all, the code of engineering is you don t implement a system with massive control before testing it.
1
SPEAKER_00
Now, perhaps the culture of engineering will radically change, then I would worry.
1
SPEAKER_00
But I don t see any signs that engineers will suddenly do idiotic things, like put an electrical power plant in control of a system that they haven t tested first.
1
SPEAKER_00
Or all of these scenarios not only imagine a
1
SPEAKER_00
almost a magically powered intelligence, including things like cure cancer, which is probably an incoherent goal because there s so many different kinds of cancer, or bring about world peace.
1
SPEAKER_00
I mean, how do you even specify that as a goal?
1
SPEAKER_00
But the scenarios also imagine some degree of control of every molecule in the universe.
1
SPEAKER_00
which not only is itself unlikely, but we would not start to connect these systems to infrastructure without testing as we would any kind of engineering system.
1
SPEAKER_00
Now, maybe some engineers will be irresponsible and we need legal and regulatory and legal responsibility implemented so that engineers don t do things that are stupid by their own standards.
1
SPEAKER_00
But the, I ve never seen enough of a plausible scenario of existential threat to devote large amounts of brain power to forestall it.
1
SPEAKER_00
That is.
1
SPEAKER_01
So you believe in the sort of the power en masse of the engineering of reason, as you argue in your latest book of Reason and Science, to sort of be the very thing that guides the development of new technology so it s safe and also keeps us safe.
1
SPEAKER_00
Yeah, it s the same, you know, granted the same culture of safety that currently is part of the engineering mindset for airplanes, for example.
1
SPEAKER_00
So yeah, I don t think that that that should be thrown out the window and that untested, all-powerful systems should be suddenly implemented.
1
SPEAKER_00
But there s no reason to think they are.
1
SPEAKER_00
And in fact, if you look at the
1
SPEAKER_00
progress of artificial intelligence, it s been impressive, especially in the last 10 years or so.
1
SPEAKER_00
But the idea that suddenly there ll be a step function, that all of a sudden, before we know it, it will be all-powerful, that there ll be some kind of recursive self-improvement, some kind of fume, is also fanciful.
1
SPEAKER_00
Certainly by the technology that now impresses us, such as deep learning, where you train something on
1
SPEAKER_00
hundreds of thousands or millions of examples.
1
SPEAKER_00
They re not hundreds of thousands of problems of which curing cancer is a typical example.
1
SPEAKER_00
And so the kind of techniques that have allowed AI to increase in the last five years are not the kind that are gonna lead to this fantasy of exponential sudden self-improvement.
1
SPEAKER_00
I think it s kind of a magical thinking.
1
SPEAKER_00
It s not based on our understanding of how AI actually works.
1
SPEAKER_01
Now, give me a chance here.
1
SPEAKER_01
So, you said fanciful, magical thinking.
1
SPEAKER_01
In his TED Talk, Sam Harris says that thinking about AI killing all human civilization is somehow fun, intellectually.
1
SPEAKER_01
Now, I have to say, as a scientist and engineer, I don t find it fun.
1
SPEAKER_01
but when I m having beer with my non-AI friends, there is indeed something fun and appealing about it, like talking about an episode of Black Mirror, considering if a large meteor is headed towards Earth, we were just told a large meteor is headed towards Earth, something like this, and can you relate to this sense of fun, and do you understand the psychology of it?
1
SPEAKER_00
Yes, right, good question.
1
SPEAKER_00
I personally don t find it fun.
1
SPEAKER_00
I find it kind of actually a waste of time, because there are genuine threats that we ought to be thinking about, like pandemics, like cybersecurity vulnerabilities, like the possibility of nuclear war, and certainly climate change.
1
SPEAKER_00
This is enough to fill many conversations without, and I think Sam did put his finger on something, namely that there is a community, sometimes called the rationality community, that delights in using its brain power to come up with scenarios that would not occur to mere mortals, to less cerebral people.
1
SPEAKER_00
So, there is a kind of intellectual thrill in finding new things to worry about that no one has worried about yet.
1
SPEAKER_00
I actually think, though, that it s – not only is it a kind of fun that doesn t give me particular pleasure, but I think there can be a pernicious side to it, namely that you overcome people with such dread, such fatalism.
1
SPEAKER_00
that there s so many ways to die, to annihilate our civilization, that we may as well enjoy life while we can.
1
SPEAKER_00
There s nothing we can do about it.
1
SPEAKER_00
If climate change doesn t do us in, then runaway robots will.
1
SPEAKER_00
So let s enjoy ourselves now.
1
SPEAKER_00
We ve got to prioritize.
1
SPEAKER_00
We have to look at threats that are close to certainty, such as climate change, and distinguish those from ones that are merely imaginable, but with infinitesimal probabilities.
1
SPEAKER_00
And we have to take into account people s worry budget.
1
SPEAKER_00
You can t worry about everything.
1
SPEAKER_00
And if you sow dread and fear and terror and fatalism, it can lead to a kind of numbness.
1
SPEAKER_00
Well, these problems are overwhelming and the engineers are just gonna kill us all.
1
SPEAKER_00
So let s either destroy the entire infrastructure of science, technology, or let s just enjoy life while we can.
1
SPEAKER_01
So there s a certain line of worry, which I m worried about a lot of things in engineering.
1
SPEAKER_01
There s a certain line of worry when you cross, you re allowed to cross, that it becomes paralyzing fear as opposed to productive fear.
1
SPEAKER_01
And that s kind of what you re highlighting.
1
SPEAKER_00
Exactly right.
1
SPEAKER_00
And we ve seen some, we know that human effort is not well calibrated against risk in that because
1
SPEAKER_00
A basic tenet of cognitive psychology is that perception of risk, and hence perception of fear, is driven by imaginability, not by data.
1
SPEAKER_00
And so we misallocate vast amounts of resources to avoiding terrorism, which kills on average about six Americans a year, with one exception of 9-11.
1
SPEAKER_00
We invade countries, we invent entire new departments of government with massive expenditure of resources and lives to defend ourselves against a trivial risk.
1
SPEAKER_00
Whereas guaranteed risks, you mentioned traffic fatalities.
1
SPEAKER_00
And even risks that are not here, but are plausible enough to worry about, like pandemics, like nuclear war, receive far too little attention.
1
SPEAKER_00
In presidential debates, there s no discussion of how to minimize the risk of nuclear war.
1
SPEAKER_00
Lots of discussion of terrorism, for example.
1
SPEAKER_00
And so I think it s essential to calibrate our
1
SPEAKER_00
budget of fear, worry, concern, planning to the actual probability of harm.
1
SPEAKER_01
Yep, so let me ask then this question.
1
SPEAKER_01
So speaking of imaginability, you said that it s important to think about reason, and one of my favorite people who likes to dip into the outskirts of reason through fascinating exploration of his imagination is Joe Rogan.
1
SPEAKER_00
Oh, yes.
1
SPEAKER_01
Who has, through reason, used to believe a lot of conspiracies, and through reason has stripped away a lot of his beliefs in that way.
1
SPEAKER_01
So it s fascinating actually to watch him, through rationality, kind of throw away the ideas of Bigfoot and 9-11.
1
SPEAKER_01
I m not sure exactly.
1
SPEAKER_00
Kim Trails, I don t know what he believes in.
1
SPEAKER_00
Yes, okay.
1
SPEAKER_00
Believed in, that s right.
1
SPEAKER_00
No, he s become a real force for good.
1
SPEAKER_01
So you were on the Joe Rogan podcast in February and had a fascinating conversation, but as far as I remember, didn t talk much about artificial intelligence.
1
SPEAKER_01
I will be on this podcast in a couple weeks.
1
SPEAKER_01
Joe is very much concerned about existential threat of AI, I m not sure if you re, which is why I was hoping that you would get into that topic.
1
SPEAKER_01
And in this way, he represents quite a lot of people who look at the topic of AI from 10,000 foot level.
1
SPEAKER_01
So as an exercise of communication, you said it s important to be rational and reason about these things.
1
SPEAKER_01
Let me ask, if you were to coach me as an AI researcher about how to speak to Joe and the general public about AI, what would you advise?
1
SPEAKER_00
Well, the short answer would be to read the sections that I wrote in Enlightenment Now about AI.
1
SPEAKER_00
But a longer reason would be, I think, to emphasize, and I think you re very well positioned as an engineer to remind people about the culture of engineering, that it really is safety-oriented.
1
SPEAKER_00
Another discussion in Enlightenment Now, I plot
1
SPEAKER_00
rates of accidental death from various causes, plane crashes, car crashes, occupational accidents, even death by lightning strikes.
1
SPEAKER_00
And they all plummet because the culture of engineering is how do you squeeze out the lethal risks, death by fire, death by drowning,
1
SPEAKER_00
death by asphyxiation, all of them drastically declined because of advances in engineering that I gotta say I did not appreciate until I saw those graphs.
1
SPEAKER_00
And it is because exactly people like you who stay up at night thinking, oh my God, what I m inventing likely to hurt people and to deploy ingenuity to prevent that from happening.
1
SPEAKER_00
Now, I m not an engineer.
1
SPEAKER_00
Although I spent 22 years at MIT, so I know something about the culture of engineering.
1
SPEAKER_00
My understanding is that this is the way you think if you re an engineer.
1
SPEAKER_00
And it s essential that that culture not be suddenly switched off when it comes to artificial intelligence.
1
SPEAKER_00
So, I mean, that could be a problem, but is there any reason to think it would be switched off?
1
SPEAKER_01
I don t think so.
1
SPEAKER_01
And one, there s not enough engineers speaking up for this way, for the excitement, for the positive view of human nature, what you re trying to create is the positivity.
1
SPEAKER_01
Like everything we try to invent is trying to do good for the world.
1
SPEAKER_01
But let me ask you about the psychology of negativity.
1
SPEAKER_01
It seems, just objectively, not considering the topic, it seems that being negative about the future makes you sound smarter than being positive about the future, irregardless of topic.
1
SPEAKER_01
Am I correct in this observation?
1
SPEAKER_01
And if so, why do you think that is?
1
SPEAKER_00
Yeah, I think there is that phenomenon.
1
SPEAKER_00
As Tom Lehrer, the satirist, said, always predict the worst and you ll be hailed as a prophet.
1
SPEAKER_00
It may be part of our overall negativity bias.
1
SPEAKER_00
We are, as a species, more attuned to the negative than the positive.
1
SPEAKER_00
We dread losses more than we enjoy gains.
1
SPEAKER_00
And that might open up a space for prophets to remind us of harms and risks and losses that we may have overlooked.
1
SPEAKER_00
So I think there is that asymmetry.
1
SPEAKER_01
So you ve written some of my favorite books all over the place, so starting from Enlightenment Now to The Better Angels of Our Nature, Blank Slate, How the Mind Works, the one about language, Language Instinct.
1
SPEAKER_01
Bill Gates, big fan too, said of your most recent book that it s my new favorite book of all time.
1
SPEAKER_01
So for you as an author, what was a book early on in your life that had a profound impact on the way you saw the world?
1
SPEAKER_00
Certainly this book, Enlightenment, now is influenced by David Deutsch s The Beginning of Infinity.
1
SPEAKER_00
A rather deep reflection on knowledge and the power of knowledge to improve the human condition.
1
SPEAKER_00
And with bits of wisdom such as that problems are inevitable, but problems are solvable, given the right knowledge, and that solutions create new problems that have to be solved in their turn.
1
SPEAKER_00
That s, I think, a kind of wisdom about the human condition that influenced the writing of this book.
1
SPEAKER_00
There s some books that are excellent but obscure, some of which I have on a page of my website.
1
SPEAKER_00
I read a book called The History of Force, self-published by a political scientist named James Paine on the historical decline of violence, and that was one of the inspirations for The Better Angels of Our Nature.
1
SPEAKER_01
What about early on?
1
SPEAKER_01
If you look back when you were maybe a teenager?
1
SPEAKER_00
I loved a book called 1, 2, 3, Infinity.
1
SPEAKER_00
When I was a young adult, I read that book by George Gamow, the physicist.
1
SPEAKER_00
He had very accessible and humorous explanations of relativity, of number theory.
1
SPEAKER_00
of dimensionality, high multiple dimensional spaces in a way that I think is still delightful 70 years after it was published.
1
SPEAKER_00
I like the Time-Life Science series.
1
SPEAKER_00
These are books that arrive every month that my mother subscribed to, each one on a different topic.
1
SPEAKER_00
One would be on electricity, one would be on forests, one would be on evolution, and then one was on the mind.
1
SPEAKER_00
And I was just intrigued that there could be a science of mind, and that book I would cite as an influence as well.
1
SPEAKER_01
So that s when you fell in love with the idea of studying the mind?
1
SPEAKER_01
Was that the thing that grabbed you?
1
SPEAKER_01
It was one of the things, I would say.
1
SPEAKER_00
I read as a college student the book Reflections on Language by Noam Chomsky, who spent most of his career here at MIT.
1
SPEAKER_00
Richard Dawkins  two books, The Blind Watchmaker and The Selfish Gene, were enormously influential, mainly for the content, but also for the writing style.
2
SPEAKER_00
ability to explain abstract concepts in lively prose.
2
SPEAKER_00
Stephen Jay Gould s first collection ever since Darwin, also an excellent example of lively writing.
2
SPEAKER_00
George Miller, a psychologist that most psychologists are familiar with.
2
SPEAKER_00
came up with the idea that human memory has a capacity of seven plus or minus two chunks.
2
SPEAKER_00
That s probably his biggest claim to fame.
2
SPEAKER_00
But he wrote a couple of books on language and communication that I read as an undergraduate.
2
SPEAKER_00
Again, beautifully written and intellectually deep.
2
SPEAKER_01
Wonderful.
2
SPEAKER_01
Stephen, thank you so much for taking the time today.
2
SPEAKER_00
My pleasure.
2
SPEAKER_00
Thanks a lot, Lex.