0
SPEAKER_01
Welcome to the Artificial Intelligence Podcast.
0
SPEAKER_01
My name is Lex Friedman.
0
SPEAKER_01
I m a research scientist at MIT.
0
SPEAKER_01
This podcast is an extension of the courses on deep learning, autonomous vehicles, and artificial general intelligence that I ve taught and organized.
0
SPEAKER_01
It is not only about machine learning, or robotics, or neuroscience, or philosophy, or any one technical field.
0
SPEAKER_01
It considers all of these avenues of thought in a way that is hopefully accessible to everyone.
0
SPEAKER_01
The aim here is to explore the nature of human and machine intelligence, the big picture of understanding the human mind and creating echoes of it in the machine.
0
SPEAKER_01
To me, that is one of our civilization s most challenging and exciting scientific journeys into the unknown.
0
SPEAKER_01
I will first repost parts of previous YouTube conversations and lecture Q&As that can be listened to without video.
0
SPEAKER_01
If you want to see the video version, please go to my YouTube channel.
0
SPEAKER_01
My username is Lex Friedman there and on Twitter, so reach out and connect if you find these conversations interesting.
0
SPEAKER_01
Moving forward, this podcast will be long form conversations with some of the most fascinating people in the world who are thinking about the nature of intelligence.
0
SPEAKER_01
But first, like I said, I will be posting old content, but now in audio form.
0
SPEAKER_01
For a little while, I ll probably repeat this intro for reposted YouTube content like this episode, and we ll try to keep it to what looks to be just over two minutes, maybe 2.30.
0
SPEAKER_01
So in the future, if you want to skip this intro, just jump to the 2.30 minute mark.
0
SPEAKER_01
In this episode, I talk with Max Tegmark.
0
SPEAKER_01
He s a professor at MIT, a physicist who has spent much of his career studying and writing about the mysteries of our cosmological universe.
0
SPEAKER_01
and now thinking and writing about the beneficial possibilities and existential risks of artificial intelligence.
0
SPEAKER_01
He s the co-founder of the Future of Life Institute, author of two books, Our Mathematical Universe and Life 3.0.
0
SPEAKER_01
He is truly an out-of-the-box thinker, so I really enjoyed this conversation.
0
SPEAKER_01
I hope you do as well.
0
SPEAKER_01
Do you think there s intelligent life out there in the universe?
0
SPEAKER_01
Let s open up with an easy question.
0
SPEAKER_00
I have a minority view here, actually.
0
SPEAKER_00
When I give public lectures, I often ask for a show of hands who thinks there s intelligent life out there somewhere else.
0
SPEAKER_00
And almost everyone puts their hands up.
0
SPEAKER_00
And when I ask why, they ll be like, oh, there s so many galaxies out there.
0
SPEAKER_00
There s got to be.
0
SPEAKER_00
But I m a numbers nerd, right?
0
SPEAKER_00
So when you look more carefully at it, it s not so clear at all.
0
SPEAKER_00
When we talk about our universe, first of all, we don t mean all of space.
0
SPEAKER_00
We actually mean, I don t know, you can throw me the universe if you want, it s behind you there.
0
SPEAKER_00
We simply mean the spherical region of space from which light has had time to reach us so far during the 13.8 billion years since our Big Bang.
0
SPEAKER_00
There s more space here, but this is what we call a universe because that s all we have access to.
0
SPEAKER_00
So is there intelligent light here that s gotten to the point of building telescopes and computers?
0
SPEAKER_00
My guess is no, actually.
0
SPEAKER_00
The probability of it happening on any given planet
0
SPEAKER_00
is some number we don t know what it is.
0
SPEAKER_00
And what we do know is that the number can t be super high because there s over a billion Earth-like planets in the Milky Way galaxy alone, many of which are billions of years older than Earth.
0
SPEAKER_00
And aside from some UFO believers, there isn t much evidence that
0
SPEAKER_00
any super advanced civilization has come here at all.
0
SPEAKER_00
So that s the famous Fermi paradox, right?
0
SPEAKER_00
And then if you work the numbers, what you find is that if you have no clue what the probability is of getting life on a given planet, so it could be 10 to the minus 10, 10 to the minus 20, or 10 to the minus 2, or any power of 10 is sort of equally likely if you want to be really open-minded.
0
SPEAKER_00
That translates into it being equally likely that our nearest neighbor is 10 to the 16 meters away, 10 to the 17 meters away, 10 to the 18.
0
SPEAKER_00
By the time you get
0
SPEAKER_00
much less than 10 to the 16 already, we pretty much know there is nothing else that close.
0
SPEAKER_00
Because they would have discovered us.
0
SPEAKER_00
Yeah, they would have discovered us long ago, or if they re really close, we would have probably noted some engineering projects that they re doing.
0
SPEAKER_00
And if it s beyond 10 to the 26 meters, that s already outside of here.
0
SPEAKER_00
So my guess is actually that
0
SPEAKER_00
We are the only life in here that s gotten to the point of building advanced tech, which I think is very...
0
SPEAKER_00
puts a lot of responsibility on our shoulders to not screw up.
0
SPEAKER_00
I think people who take for granted that it s okay for us to screw up, have an accidental nuclear war, or go extinct somehow because there s a sort of Star Trek-like situation out there where some other life forms are going to come and bail us out and it doesn t matter as much.
0
SPEAKER_00
I think they re lulling us into a false sense of security.
0
SPEAKER_00
I think it s much more prudent to say, let s be really grateful for this amazing opportunity we ve had.
0
SPEAKER_00
make the best of it, just in case it is down to us.
0
SPEAKER_01
So from a physics perspective, do you think intelligent life, so it s unique from a sort of statistical view of the size of the universe, but from the basic matter of the universe, how difficult is it for intelligent life to come about, the kind of advanced tech building life?
0
SPEAKER_01
is implied in your statement that it s really difficult to create something like a human species?
0
SPEAKER_00
Well, I think what we know is that going from no life to having life that can do a level of tech, there s some sort of, to going beyond that and actually settling our whole universe with life, there s some road, major roadblock
0
SPEAKER_00
there, which is some great filter as it s sometimes called, which is tough to get through.
0
SPEAKER_00
It s either, that roadblock is either behind us or in front of us.
0
SPEAKER_00
I m hoping very much that it s behind us.
0
SPEAKER_00
I m super excited every time we get a new report from NASA saying they failed to find any life on Mars.
0
SPEAKER_00
Like, yes, awesome.
0
SPEAKER_00
Because that suggests that the hard part, maybe it was getting the first ribosome or some very low level kind of stepping stone so that we re home free.
0
SPEAKER_00
Because if that s true, then the future is really only limited by our own imagination.
0
SPEAKER_00
It would be much suckier if it turns out that
0
SPEAKER_00
this level of life is kind of a dime a dozen but maybe there s some other problem like as soon as civilization gets advanced technology within a hundred years they get into some stupid fight with themselves and poof!
0
SPEAKER_00
That would be a bummer.
0
SPEAKER_01
Yeah, so you ve explored the mysteries of the universe, the cosmological universe, the one that s sitting between us today.
0
SPEAKER_01
I think you ve also begun to explore the other universe, which is sort of the mystery, the mysterious universe of the mind, of intelligence, of intelligent life.
0
SPEAKER_01
So is there a common thread between your interest and the way you think about space and intelligence?
0
SPEAKER_00
Oh yeah.
0
SPEAKER_00
When I was a teenager,
0
SPEAKER_00
I was already very fascinated by the biggest questions and I felt that the two biggest mysteries of all in science were our universe out there and our universe in here.
0
SPEAKER_00
So it s quite natural after having spent a quarter of a century.
0
SPEAKER_00
on my career thinking a lot about this one, and now indulging in the luxury of doing research on this one.
0
SPEAKER_00
It s just so cool.
0
SPEAKER_00
I feel the time is ripe now for greatly deepening our understanding of this.
0
SPEAKER_00
To start exploring this one.
0
SPEAKER_00
Yeah, because I think a lot of people view intelligence as
0
SPEAKER_00
something mysterious that can only exist in biological organisms like us, and therefore dismiss all talk about artificial general intelligence as science fiction.
0
SPEAKER_00
But from my perspective as a physicist, I am a blob of quarks and electrons moving around in a certain pattern and processing information in certain ways.
0
SPEAKER_00
This is also a blob of quarks and electrons.
0
SPEAKER_00
I m not smarter than the water bottle because I m made of different kind of quarks.
0
SPEAKER_00
I m made of up quarks and down quarks, exact same kind as this.
0
SPEAKER_00
There s no secret sauce, I think, in me.
0
SPEAKER_00
It s all about the pattern of the information processing.
0
SPEAKER_00
And this means that there s no law of physics saying that we can t create technology which can help.
0
SPEAKER_00
us by being incredibly intelligent and help us crack mysteries that we couldn t.
0
SPEAKER_00
In other words, I think we ve really only seen the tip of the intelligence iceberg so far.
0
SPEAKER_01
Yeah, so the perceptronium.
0
SPEAKER_01
Yeah.
0
SPEAKER_01
So you coined this amazing term.
0
SPEAKER_01
It s a hypothetical state of matter, sort of thinking from a physics perspective, what is the kind of matter that can help, as you re saying, subjective experience emerge, consciousness emerge.
0
SPEAKER_01
So how do you think about consciousness from this physics perspective?
0
SPEAKER_00
Very good question.
0
SPEAKER_00
So again, I think many people have underestimated our ability to make progress on this by convincing themselves it s hopeless because somehow we re missing some ingredient that we need or some new consciousness particle or whatever.
0
SPEAKER_00
I happen to think that we re not missing anything and the interesting thing about consciousness that gives us this amazing subjective experience of colors and sounds and emotions is rather something at the higher level about the patterns of information processing.
0
SPEAKER_00
That s why I like to think about this idea of perceptronium.
0
SPEAKER_00
What does it mean for an arbitrary physical system to be conscious in terms of what its particles are doing or its information is doing?
0
SPEAKER_00
I hate the carbon chauvinism, you know, this attitude you have to be made of carbon atoms to be smart or conscious.
0
SPEAKER_01
There s something about the information processing that this kind of matter performs.
0
SPEAKER_00
Yeah, and you know, you can see I have my favorite equations here describing various fundamental aspects of the world.
0
SPEAKER_00
I feel that, I think one day, maybe someone who s watching this will come up with the equations that information processing has to satisfy to be conscious.
0
SPEAKER_00
I m quite convinced there is big discovery to be made there.
0
SPEAKER_00
Because, let s face it, we know that some information processing is conscious because
0
SPEAKER_00
We are conscious, but we also know that a lot of information processing is not conscious, like most of the information processing happening in your brain right now.
0
SPEAKER_00
is not conscious, they re like 10 megabytes per second coming in, even just through your visual system.
0
SPEAKER_00
You re not conscious about your heartbeat regulation or most things.
0
SPEAKER_00
Even if I just ask you to read what it says here, you look at it and then, oh, now you know what it said, but you re not aware of how the computation actually happened.
0
SPEAKER_00
Your consciousness is like the CEO that got an email at the end with the final answer.
0
SPEAKER_00
What is it that makes a difference?
0
SPEAKER_00
I think that s both a great science mystery.
0
SPEAKER_00
We re actually studying it a little bit in my lab here at MIT.
0
SPEAKER_00
But I also think it s just a really urgent question to answer.
0
SPEAKER_00
For starters, I mean, if you re an emergency room doctor and you have an unresponsive patient coming in, wouldn t it be great if in addition to having
0
SPEAKER_00
a CT scanner, you had a consciousness scanner that could figure out whether this person is actually having locked in syndrome or is actually comatose.
0
SPEAKER_00
And in the future, imagine if we build robots or the machine that
0
SPEAKER_00
we can have really good conversations with, which I think is most likely to happen, right?
0
SPEAKER_00
Wouldn t you want to know like if your home helper robot is actually experiencing anything or just like a zombie?
0
SPEAKER_00
I mean, would you prefer it?
0
SPEAKER_00
What would you prefer?
0
SPEAKER_00
Would you prefer that it s actually unconscious so that you don t have to feel guilty about switching it off or giving boring chores?
0
SPEAKER_00
Or what would you prefer?
0
SPEAKER_01
Well, certainly we would prefer, I would prefer the appearance of consciousness.
0
SPEAKER_01
But the question is whether the appearance of consciousness is different than consciousness itself.
0
SPEAKER_01
And sort of to ask that as a question, do you think we need to understand what consciousness is, solve the hard problem of consciousness in order to build something like an AGI system?
0
SPEAKER_01
No.
0
SPEAKER_00
I don t think that.
0
SPEAKER_00
I think we will probably be able to build things even if we don t answer that question.
0
SPEAKER_00
But if we want to make sure that what happens is a good thing, we better solve it first.
0
SPEAKER_00
So it s a wonderful controversy you re raising there, where you have basically three points of view about the heart problem.
0
SPEAKER_00
So there are two different points of view that both conclude that the heart problem of consciousness is BS.
0
SPEAKER_00
On one hand, you have some people like Daniel Dennett, who say consciousness is just BS, because consciousness is the same thing as intelligence.
0
SPEAKER_00
There s no difference.
0
SPEAKER_00
So anything which acts conscious is conscious, just like we are.
0
SPEAKER_00
And then there are also a lot of people, including many top AI researchers I know, who say, oh, conscience is just bullshit because, of course, machines can never be conscious.
0
SPEAKER_00
They re always going to be zombies.
0
SPEAKER_00
You never have to feel guilty about how you treat them.
0
SPEAKER_00
And then there s a third group of people, including Giulio Tononi, for example, and Christoph Koch and a number of others.
0
SPEAKER_00
I would put myself on this middle camp who say that actually some information processing is conscious and some is not.
0
SPEAKER_00
So let s find the equation, which can be used to determine which it is.
0
SPEAKER_00
And I think we ve just been a little bit.
0
SPEAKER_00
lazy kind of running away from this problem for a long time.
0
SPEAKER_00
It s been almost taboo to even mention the C word in a lot of circles.
0
SPEAKER_00
But we should stop making excuses.
0
SPEAKER_00
This is a science question and there are ways we can even test any theory that makes predictions for this.
0
SPEAKER_00
And coming back to this helper robot, I mean, so you said you d want your helper robot to certainly act conscious and treat you like, have conversations with you and stuff.
0
SPEAKER_00
I think so.
0
SPEAKER_00
But wouldn t you, would you feel, would you feel a little bit creeped out if you realized that it was just.
0
SPEAKER_00
a glossed up tape recorder that was just zombie and was a faking emotion.
0
SPEAKER_00
Would you prefer that it actually had an experience or would you prefer that it s actually not experiencing anything so you don t have to feel guilty about what you do to it?
0
SPEAKER_01
It s such a difficult question because
0
SPEAKER_01
You know, it s like when you re in a relationship and you say, well, I love you.
0
SPEAKER_01
And the other person says, I love you back.
0
SPEAKER_01
It s like asking, well, do they really love you back?
0
SPEAKER_01
Or are they just saying they love you back?
0
SPEAKER_01
Don t you really want them to actually love you?
0
SPEAKER_01
It s hard to really know the difference between
0
SPEAKER_01
everything seeming like there s consciousness present, there s intelligence present, there s affection, passion, love, and it actually being there.
0
SPEAKER_01
I m not sure.
0
SPEAKER_00
Can I ask you a question about this?
0
SPEAKER_00
To make it a bit more pointed.
0
SPEAKER_00
Mass General Hospital is right across the river, right?
0
SPEAKER_00
Yes.
0
SPEAKER_00
Suppose you re going in for a medical procedure.
0
SPEAKER_00
And they re like, you know, for anesthesia, what we re going to do is we re going to give you muscle relaxants so you won t be able to move.
0
SPEAKER_00
And you re going to feel excruciating pain during the whole surgery, but you won t be able to do anything about it.
0
SPEAKER_00
But then we re going to give you this drug that erases your memory of it.
0
SPEAKER_00
Would you be cool about that?
0
SPEAKER_00
What s the difference that you re conscious about it or not if there s no behavioral change, right?
0
SPEAKER_01
Right.
0
SPEAKER_01
That s a really clear way to put it.
0
SPEAKER_01
Yeah, it feels like in that sense experiencing it is a valuable quality.
0
SPEAKER_01
So actually being able to have subjective experiences, at least in that case, is valuable.
0
SPEAKER_00
And I think we humans have a little bit of a bad track record also of making these self-serving arguments that other entities aren t conscious.
0
SPEAKER_00
You know, people often say, Oh, these animals can t feel pain.
0
SPEAKER_00
It s okay to boil lobsters because we asked them if it hurt and they didn t say anything.
0
SPEAKER_00
And now there was just a paper out saying lobsters did do feel pain when you boil them and they re banning it in Switzerland.
0
SPEAKER_00
And, and we did this with slaves too often and said, Oh, they don t mind.
0
SPEAKER_00
They don t maybe.
0
SPEAKER_00
or can t conscious or women don t have souls or whatever.
0
SPEAKER_00
So I m a little bit nervous when I hear people just take as an axiom that machines can t have experience ever.
0
SPEAKER_00
I think this is just this really fascinating science question is what it is.
0
SPEAKER_00
Let s research it and try to figure out what it is that makes the difference between unconscious intelligent behavior and conscious intelligent behavior.
0
SPEAKER_01
So in terms of, so if you think of it, Boston Dynamics humanoid robot being sort of with a broom being pushed around, it starts pushing on this consciousness question.
0
SPEAKER_01
So let me ask, do you think an AGI system, like a few neuroscientists believe, needs to have a physical embodiment, needs to have a body or something like a body?
0
SPEAKER_00
No.
0
SPEAKER_01
I don t think so.
0
SPEAKER_00
You mean to have a conscious experience?
0
SPEAKER_00
To have consciousness.
0
SPEAKER_00
I do think it helps a lot to have a physical embodiment to learn the kind of things about the world that are important to us humans, for sure.
0
SPEAKER_00
But I don t think the physical embodiment is necessary.
1
SPEAKER_02
after you ve learned it, just have the experience.
1
SPEAKER_02
Think about when you re dreaming, right?
1
SPEAKER_02
Your eyes are closed, you re not getting any sensory input, you re not behaving or moving in any way, but there s still an experience there, right?
1
SPEAKER_02
And so clearly the experience that you have when you see something cool in your dreams isn t coming from your eyes, it s just the information processing itself in your brain, which is that experience, right?
1
SPEAKER_01
But if I put it another way, I ll say because it comes from neuroscience, is the reason you wanna have a body and a physical, something like a physical, like a physical system, is because you want to be able to preserve something.
1
SPEAKER_01
In order to have a self, you could argue, you need to have some kind of embodiment of self to want to preserve.
1
SPEAKER_02
Well, now we re getting a little bit anthropomorphic, anthropomorphizing things, maybe talking about self-preservation instincts.
1
SPEAKER_02
I mean, we are evolved organisms, right?
1
SPEAKER_02
So Darwinian evolution endowed us and other evolved organism with a self-preservation instinct because those
1
SPEAKER_02
that didn t have those self-preservation genes got cleaned out of the gene pool.
1
SPEAKER_02
But if you build an artificial general intelligence, the mind space that you can design is much, much larger than just a specific subset of minds that can evolve.
1
SPEAKER_02
So an AGI mind doesn t necessarily have to have any self-preservation instinct
1
SPEAKER_02
It also doesn t necessarily have to be so individualistic as us.
1
SPEAKER_02
Like imagine if you could just, first of all, we re also very afraid of death.
1
SPEAKER_02
You know, I suppose you could back yourself up every five minutes and then your airplane is about to crash.
1
SPEAKER_02
You re like, shucks, I m just, I m dead.
1
SPEAKER_02
I m gonna lose the last five minutes of experiences since my last cloud backup.
1
SPEAKER_02
It s not as big a deal.
1
SPEAKER_02
Or if we could just copy experiences between our minds easily, which we could easily do if we were silicon-based, then maybe we would feel a little bit more like a hive mind.
1
SPEAKER_02
So I don t think we should take for granted at all that AGI will have to have any of those
1
SPEAKER_02
competitive as alpha male instincts.
1
SPEAKER_02
On the other hand, you know, this is really interesting because I think some people go too far and say, of course, we don t have to have any concerns either that advanced AI will have those instincts because we can build anything we want.
1
SPEAKER_02
That there s a very nice set of arguments going back to Steve Omohundro and Nick Bostrom and others just pointing out that when we build machines,
1
SPEAKER_02
We normally build them with some kind of goal, you know, win this chess game, drive this car safely or whatever.
1
SPEAKER_02
And as soon as you put in a goal into machine, especially if it s kind of open-ended goal and the machine is very intelligent, it ll break that down into a bunch of sub goals.
1
SPEAKER_02
And one of those goals will almost always be self-preservation because if it breaks or dies in the process, it s not going to accomplish the goal, right?
1
SPEAKER_02
Like suppose you just build a little, you have a little robot and you tell it to go down.
1
SPEAKER_02
the star market here and get you some food, make you cook an Italian dinner, you know.
1
SPEAKER_02
And then someone mugs it and tries to break it on the way.
1
SPEAKER_02
That robot has an incentive to not get destroyed and defend itself or run away, because otherwise it s going to fail in cooking you dinner.
1
SPEAKER_02
It s not afraid of death, but it really wants to complete the dinner cooking goal, so it will have a self-preservation instinct.
1
SPEAKER_02
Yeah, continue being a functional agent somehow.
1
SPEAKER_02
Yeah.
1
SPEAKER_02
If you give any kind of more ambitious goal to an AGI, it s very likely they want to acquire more resources so it can do that better.
1
SPEAKER_02
And it s exactly from those sort of sub-goals that we might not have intended that some of the concerns about AGI safety come.
1
SPEAKER_02
You give it some goal that seems completely harmless and then
1
SPEAKER_02
before you realize it, it s also trying to do these other things which you didn t want it to do and it s maybe smarter than us.
1
SPEAKER_02
So it s fascinating.
1
SPEAKER_01
And let me pause just because I am in a very kind of human-centric way see fear of death as a valuable motivator.
1
SPEAKER_01
So you don t think
1
SPEAKER_01
You think that s an artifact of evolution, so that s the kind of mind space evolution created that we re sort of almost obsessed about self-preservation at some kind of genetic level.
1
SPEAKER_01
You don t think that s necessary to be afraid of death?
1
SPEAKER_01
So not just a kind of sub-goal of self-preservation just so you can keep doing the thing, but more fundamentally sort of have the finite thing, like this ends for you at some point.
1
SPEAKER_01
Interesting.
1
SPEAKER_02
Do I think it s necessary for what precisely?
1
SPEAKER_01
For intelligence, but also for consciousness.
1
SPEAKER_01
So for both.
1
SPEAKER_01
Do you think really like a finite death and the fear of it is important?
1
SPEAKER_02
Before I can answer, before we can agree on whether it s necessary for intelligence or for conscience, we should be clear on how we define those two words because a lot of really smart people define them in very different ways.
1
SPEAKER_02
I was in this, on this panel with AI experts and they couldn t, they couldn t agree on how to define intelligence even.
1
SPEAKER_02
So I define intelligence simply as the ability to accomplish complex goals.
1
SPEAKER_02
I like your broad definition because again, I don t want to be a carbon chauvinist.
1
SPEAKER_02
Right.
1
SPEAKER_02
And, um,
1
SPEAKER_02
In that case, no, certainly it doesn t require fear of death.
1
SPEAKER_02
I would say AlphaGo or AlphaZero is quite intelligent.
1
SPEAKER_02
I don t think AlphaZero has any fear of being turned off because it doesn t understand the concept of it even.
1
SPEAKER_02
And similarly, consciousness, I mean, you can certainly imagine
1
SPEAKER_02
It s a very simple kind of experience.
1
SPEAKER_02
If certain plants have any kind of experience, I don t think they re very afraid of dying.
1
SPEAKER_02
There s nothing they can do about it anyway.
1
SPEAKER_02
So there wasn t that much value.
1
SPEAKER_02
But more seriously, I think if you ask not just about being conscious, but maybe having what we might call an exciting life where you feel passion and really appreciate
1
SPEAKER_02
Maybe there perhaps it does help having a backdrop that hey, it s finite, let s make the most of this, let s live to the fullest.
1
SPEAKER_02
But if you knew you were going to just live forever, do you think you would change your
1
SPEAKER_01
Yeah, I mean, in some perspective, it would be an incredibly boring life, living forever.
1
SPEAKER_01
So in the sort of loose, subjective terms that you said of something exciting and something that other humans would understand, I think, is yeah, it seems that the finiteness of it is important.
1
SPEAKER_02
Well, the good news I have for you then is, based on what we understand about cosmology, everything in our universe is ultimately probably finite, although... Big crunch or big... what s the infinite expansion?
1
SPEAKER_02
Yeah, we could have a big chill or a big crunch or a big rip or the big snap or death bubbles.
1
SPEAKER_02
All of them are more than a billion years away, so we certainly have vastly more time than our ancestors thought.
1
SPEAKER_02
But it s still pretty hard to squeeze in an infinite number of compute cycles, even though there are some loopholes that just might be possible.
1
SPEAKER_02
But I think some people like to say that you should live as if
1
SPEAKER_02
you re about to die in five years or so and that s sort of optimal.
1
SPEAKER_02
Maybe we, it s a good assumption, we should build our civilization as if it s all finite to be on the safe side.
1
SPEAKER_01
Right, exactly.
1
SPEAKER_01
So you mentioned defining intelligence as
1
SPEAKER_01
and the ability to solve complex goals.
1
SPEAKER_01
Where would you draw a line, or how would you try to define human level intelligence and superhuman level intelligence?
1
SPEAKER_01
Is consciousness part of that definition?
1
SPEAKER_01
No.
1
SPEAKER_02
Consciousness does not come into this definition.
1
SPEAKER_02
So I think of intelligence, it s a spectrum, but there are very many different kinds of goals you can have.
1
SPEAKER_02
You can have a goal to be a good chess player, a good go player, a good car driver, a good investor, good poet, et cetera.
1
SPEAKER_02
So intelligence by its very nature isn t something you can measure by just one number or some overall goodness.
1
SPEAKER_02
No, no.
1
SPEAKER_02
There are some people who are better at this.
1
SPEAKER_02
Some people are better at that.
1
SPEAKER_02
Right now we have machines that are much better than us at some very narrow tasks like
1
SPEAKER_02
multiplying large numbers fast, memorizing large databases, playing chess, playing Go, and soon driving cars.
1
SPEAKER_02
But there s still no machine that can match a human child in general intelligence.
1
SPEAKER_02
But artificial general intelligence, AGI, the name of your course, of course, that is
1
SPEAKER_02
by its very definition, the quest to build a machine that can do everything as well as we can.
1
SPEAKER_02
So the old holy grail of AI from back to its inception in the 60s.
1
SPEAKER_02
If that ever happens, of course, I think it s going to be the biggest transition in the history of life on earth.
1
SPEAKER_02
But it doesn t necessarily have to wait
1
SPEAKER_02
the big impact until machines are better than us at knitting, the really big change doesn t come exactly at the moment they re better than us at everything.
1
SPEAKER_02
The really big change comes, first their big change is when they start becoming better at us at doing most of the jobs that we do, because that takes away much of the demand for human labor.
1
SPEAKER_02
And then the really whopping change comes when they become better than us at AI research.
1
SPEAKER_02
Because right now, the timescale of AI research is limited by the human research and development cycle of years, typically.
1
SPEAKER_02
How long does it take from one release of some software or iPhone or whatever to the next?
1
SPEAKER_02
But once Google can replace 40,000 engineers by 40,000 equivalent
1
SPEAKER_02
pieces of software or whatever, but then there s no reason that has to be years.
1
SPEAKER_02
It can be in principle much faster.
1
SPEAKER_02
And the timescale of future progress in AI and all of science and technology will be driven by machines, not humans.
1
SPEAKER_02
So it s this simple point which
1
SPEAKER_02
gives rise to this incredibly fun controversy about whether there can be intelligence explosion, so-called singularities, as Werner Wieners called it.
1
SPEAKER_02
The idea is articulated by I.J.
1
SPEAKER_02
Goode, obviously way back in the 50s, but you can see Alan Turing and others thought about it even earlier.
1
SPEAKER_02
You asked me exactly what I define human level as.
1
SPEAKER_02
So the glib answer is to say something which is better than us at all cognitive tasks, better than any human at all cognitive tasks.
1
SPEAKER_02
But the really interesting bar, I think, goes a little bit lower than that, actually.
1
SPEAKER_02
It s when they re better than us at AI programming and general learning so that they can, if they want to,
1
SPEAKER_02
get better than us at anything by just studying up.
1
SPEAKER_01
So their better is a key word and better is towards this kind of spectrum of the complexity of goals it s able to accomplish.
1
SPEAKER_01
Yeah.
1
SPEAKER_01
So another way to, so another, and that s certainly a very clear definition of human love.
1
SPEAKER_01
So there s, it s almost like a sea that s rising.
1
SPEAKER_01
You could do more and more and more things.
1
SPEAKER_01
It s actually a graphic that you show.
1
SPEAKER_01
It s a really nice way to put it.
1
SPEAKER_01
So there s some peaks and there s an ocean level elevating and you solve more and more problems.
1
SPEAKER_01
But, you know, just kind of to take a pause and we took a bunch of questions and a lot of social networks and a bunch of people asked sort of a slightly different direction on creativity and on things that perhaps aren t a peak.
1
SPEAKER_01
It s, you know, human beings are flawed, and perhaps better means having contradiction, being flawed in some way.
1
SPEAKER_01
So let me sort of start easy, first of all.
1
SPEAKER_01
So you have a lot of cool equations.
1
SPEAKER_01
Let me ask, what s your favorite equation, first of all?
1
SPEAKER_01
I know they re all like your children, but which one is that?
1
SPEAKER_02
It s the master key of quantum mechanics of the micro world.
1
SPEAKER_02
So this, with this equation, we can calculate everything to do with atoms, molecules, and all the way up.
1
SPEAKER_01
Okay, so quantum mechanics is certainly a beautiful, mysterious formulation of our world.
1
SPEAKER_01
So I d like to sort of ask you, just as an example, it perhaps doesn t have the same beauty as physics does, but in mathematics, abstract, Andrew Wiles, who proved Fermat s last theorem,
1
SPEAKER_01
So I just saw this recently and it kind of caught my eye a little bit.
1
SPEAKER_01
This is 358 years after it was conjectured.
1
SPEAKER_01
So this very simple formulation, everybody tried to prove it, everybody failed.
1
SPEAKER_01
And so here s this guy comes along and eventually proves it and then fails to prove it and then proves it again in 94.
1
SPEAKER_01
And he said like the moment when everything connected into place,
1
SPEAKER_01
In an interview he said, it was so indescribably beautiful.
1
SPEAKER_01
That moment when you finally realize the connecting piece of two conjectures.
1
SPEAKER_01
He said, it was so indescribably beautiful.
1
SPEAKER_01
It was so simple and so elegant.
1
SPEAKER_01
I couldn t understand how I d missed it and I just stared at it in disbelief for 20 minutes.
1
SPEAKER_01
Then during the day I walked around the department and I d keep coming back to my desk looking to see if it was still there.
1
SPEAKER_01
It was still there.
1
SPEAKER_01
I couldn t contain myself.
1
SPEAKER_01
I was so excited.
1
SPEAKER_01
It was the most important moment of my working life.
1
SPEAKER_01
Nothing I ever do again will mean as much.
1
SPEAKER_01
So that particular moment, and it kind of made me think of what would it take, and I think we have all been there at small levels.
1
SPEAKER_01
Maybe let me ask, have you had a moment like that in your life where you just had an idea, it s like, wow, yes.
1
SPEAKER_02
I wouldn t mention myself in the same breath as Andrew Wiles, but I ve certainly had a number of aha moments when I
1
SPEAKER_02
realized something very cool about physics.
1
SPEAKER_02
It just completely made my head explode.
1
SPEAKER_02
In fact, some of my favorite discoveries I made, I later realized that they had been discovered earlier by someone who sometimes got quite famous for it.
1
SPEAKER_02
So it s too late for me to even publish it.
1
SPEAKER_02
But that doesn t diminish in any way the emotional experience you have when you realize it.
1
SPEAKER_02
Wow.
1
SPEAKER_01
Yeah.
1
SPEAKER_01
So what would it take in that moment, that wow, that was yours in that moment.
1
SPEAKER_01
So what do you think it takes
1
SPEAKER_01
for an intelligent system, an AGI system, an AI system, to have a moment like that?
1
SPEAKER_02
It s a tricky question because there are actually two parts to it, right?
1
SPEAKER_02
One of them is, can it accomplish that proof?
1
SPEAKER_02
Can it prove that you can never write A to the N plus B to the N equals Z to the N for all integers, et cetera, et cetera, when N is bigger than 2?
1
SPEAKER_02
That s simply a question about intelligence.
1
SPEAKER_02
Can you build machines that are that intelligent?
1
SPEAKER_02
And I think by the time we get a machine that can independently come up with that level of proofs, probably quite close to AGI.
1
SPEAKER_02
The second question is a question about consciousness.
1
SPEAKER_02
How likely is it that such a machine would actually have any experience at all, as opposed to just being like a zombie?
1
SPEAKER_02
would we expect it to have some sort of emotional response to this or anything at all akin to human emotion where when it accomplishes its machine goal, it views it as somehow something very positive and sublime and deeply meaningful.
1
SPEAKER_02
I would certainly hope that if in the future we do create machines that are our peers,
1
SPEAKER_02
or even our descendants.
1
SPEAKER_02
I would certainly hope that they do have this sort of sublime appreciation of life.
1
SPEAKER_02
In a way, my absolutely worst nightmare would be that at some point in the future, the distant future, maybe our cosmos is teeming with all this post-biological life doing all this seemingly cool stuff.
1
SPEAKER_02
And maybe the last humans, by the time our species eventually fizzles out, we ll be like, well, that s OK, because we re so proud of our descendants here.
1
SPEAKER_02
My worst nightmare is that we haven t solved the consciousness problem, and we haven t realized that these are all the zombies.
1
SPEAKER_02
They re not aware of anything anymore than the tape recorder hasn t any kind of experience.
1
SPEAKER_02
So the whole thing has just become a play for empty benches.
1
SPEAKER_02
That would be like the ultimate zombie apocalypse.
1
SPEAKER_02
So I would much rather, in that case, that we have these beings which can really appreciate
1
SPEAKER_02
how amazing it is.
1
SPEAKER_01
And in that picture, what would be the role of creativity?
1
SPEAKER_01
A few people ask about creativity.
1
SPEAKER_01
When you think about intelligence, certainly the story you told at the beginning of your book involved creating movies and so on.
1
SPEAKER_01
Making money, you know, you can make a lot of money in our modern world with music and movies, so if you are an intelligent system, you may want to get good at that.
1
SPEAKER_01
But that s not necessarily what I mean by creativity.
1
SPEAKER_01
Is it important on that complex goals where the sea is rising for there to be something creative, or am I being very human-centric and thinking creativity is somehow special relative
2
SPEAKER_01
to intelligence?
2
SPEAKER_01
My hunch is that we should think of creativity simply as an aspect of intelligence.
2
SPEAKER_01
And we have to be very careful with human vanity.
2
SPEAKER_01
We have this tendency to very often want to say, as soon as machines can do something, we try to diminish it and say, oh, but that s not like real intelligence.
2
SPEAKER_01
Because they re not creative or this or that.
2
SPEAKER_01
If we ask ourselves to write down a definition of what we actually mean by being creative, what we mean by Andrew Wiles, what he did there for example, don t we often mean that someone takes a very unexpected leap?
2
SPEAKER_01
It s not like taking 573 and multiplying it by 224 by just a step of
2
SPEAKER_01
straightforward cookbook-like rules, right?
2
SPEAKER_01
Maybe you make a connection between two things that people had never thought was connected.
2
SPEAKER_01
It s very surprising.
2
SPEAKER_01
Or something like that.
2
SPEAKER_01
I think this is an aspect of intelligence, and this is actually one of the most important aspects of it.
2
SPEAKER_01
Maybe the reason we humans tend to be better at it than traditional computers is because it s something that comes more naturally if you re a neural network than if you re a traditional logic gate-based computer machine.
2
SPEAKER_01
We physically have all these connections.
2
SPEAKER_01
And if you activate here, activate here, activate here, ping!
2
SPEAKER_01
My hunch is that if we ever build a machine
2
SPEAKER_01
where you could just give it the task, hey, you say, hey, I just realized I want to travel around the world instead this month.
2
SPEAKER_01
Can you teach my AGI course for me?
2
SPEAKER_01
And it s like, okay, I ll do it.
2
SPEAKER_01
And it does everything that you would have done and improvises and stuff.
2
SPEAKER_01
That would, in my mind, involve a lot of creativity.
2
SPEAKER_00
Yeah, so it s actually a beautiful way to put it.
2
SPEAKER_00
I think we do try to grasp at the definition of intelligence is everything we don t understand how to build.
2
SPEAKER_00
So we as humans try to find things that we have and machines don t have.
2
SPEAKER_00
And maybe creativity is just one of the things, one of the words we use to describe that.
2
SPEAKER_00
That s a really interesting way to put it.
2
SPEAKER_01
I don t think we need to be that defensive.
2
SPEAKER_01
I don t think anything good comes out of saying we re somehow special.
2
SPEAKER_01
Contrary-wise, there are many examples in history of where trying to pretend that we re somehow superior to all other intelligent beings has led to pretty bad results, right?
2
SPEAKER_01
Nazi Germany, they said that they were somehow superior to other people.
2
SPEAKER_01
Today, we still do a lot of cruelty to animals by saying that we re so superior somehow and they can t feel pain.
2
SPEAKER_01
Slavery was justified by the same kind of really weak arguments.
2
SPEAKER_01
And I don t think if we actually go ahead and build artificial general intelligence, we can do things better than us.
2
SPEAKER_01
I don t think we should try to found our self-worth on some sort of
2
SPEAKER_01
bogus claims of superiority in terms of our intelligence, I think we should instead find our calling and the meaning of life from the experiences that we have.
2
SPEAKER_01
I can have very meaningful experiences even if there are other people who are smarter than me.
2
SPEAKER_01
When I go to a faculty meeting here and we re talking about something and then I suddenly realize, oh boy, he has an old prize, he has an old prize, he has an old prize.
2
SPEAKER_01
I don t have one.
2
SPEAKER_01
Does that make me enjoy life any less or enjoy talking to those people less?
2
SPEAKER_01
Of course not.
2
SPEAKER_01
And contrary wise, I feel very honored and privileged to get to interact with other very intelligent beings that are better than me at a lot of stuff.
2
SPEAKER_01
I don t think there s any reason why we can t have the same approach with intelligent machines.
2
SPEAKER_00
That s a really interesting, so people don t often think about that.
2
SPEAKER_00
They think about when there s going, if there s machines that are more intelligent, you naturally think that that s not going to be a beneficial type of intelligence.
2
SPEAKER_00
you don t realize it could be, you know, like peers with Nobel Prizes that would be just fun to talk with and they might be clever about certain topics and you can have fun having a few drinks with them.
2
SPEAKER_01
Another example we can all relate to of why it doesn t have to be a terrible thing to be in the presence of people who are even smarter than us all around is when you and I were both two years old, I mean, our parents were much more intelligent than us, right?
2
SPEAKER_01
Worked out okay.
2
SPEAKER_01
Because their goals were aligned with our goals.
2
SPEAKER_01
And that I think is really the number one key
2
SPEAKER_01
issue we have to solve.
2
SPEAKER_01
The value alignment problem, exactly.
2
SPEAKER_01
Because people who see too many Hollywood movies with lousy science fiction plot lines, they worry about the wrong thing, right?
2
SPEAKER_01
They worry about some machines suddenly turning evil.
2
SPEAKER_01
It s not malice that s the concern, it s competence.
2
SPEAKER_01
By definition,
2
SPEAKER_01
Intelligent makes you very competent.
2
SPEAKER_01
If you have a more intelligent goal playing, computer playing is the less intelligent one.
2
SPEAKER_01
And when we define intelligence as the ability to accomplish goal winning, right?
2
SPEAKER_01
It s going to be the more intelligent one that wins.
2
SPEAKER_01
And if you have a human and then you have an AGI that s more intelligent in all ways, and they have different goals, guess who s going to get their way, right?
2
SPEAKER_01
So I was just reading about this.
2
SPEAKER_01
particular rhinoceros species that was driven extinct just a few years ago, and a bummer.
2
SPEAKER_01
I was looking at this cute picture of a mommy rhinoceros with its child, you know, and why did we humans drive it to extinction?
2
SPEAKER_01
It wasn t because we were evil rhino haters as a whole.
2
SPEAKER_01
It was just because our goals weren t aligned with those of the rhinoceros and it didn t work out so well for the rhinoceros because we were more intelligent, right?
2
SPEAKER_01
So I think it s just so important that if we ever do build AGI,
2
SPEAKER_01
Before we unleash anything, we have to make sure that it learns to understand our goals, that it adopts our goals, and it retains those goals.
2
SPEAKER_00
So the cool, interesting problem there is us as human beings trying to formulate our values.
2
SPEAKER_00
So, you know, you could think of the United States Constitution as a way that people sat down, at the time a bunch of white men, but, which is a good example, I should, we should say.
2
SPEAKER_00
They formulated the goals for this country and a lot of people agree that those goals actually held up pretty well.
2
SPEAKER_00
It s an interesting formulation of values and failed miserably in other ways.
2
SPEAKER_00
So, for the value alignment problem and the solution to it, we have to be able to
2
SPEAKER_00
put on paper or in a program human values.
2
SPEAKER_00
How difficult do you think that is?
2
SPEAKER_01
Very.
2
SPEAKER_01
But it s so important.
2
SPEAKER_01
We really have to give it our best.
2
SPEAKER_01
And it s difficult for two separate reasons.
2
SPEAKER_01
There s the technical value alignment problem of figuring out just how to make
2
SPEAKER_01
Machines understand their goals, adopt them and retain them.
2
SPEAKER_01
And then there s the separate part of it, the philosophical part, whose values anyway?
2
SPEAKER_01
And since we it s not like we have any great consensus on this planet on values.
2
SPEAKER_01
How what mechanism should we create then to aggregate and decide, OK, what s a good compromise?
2
SPEAKER_01
Right.
2
SPEAKER_01
That second discussion can t just be left the tech nerds like myself.
2
SPEAKER_01
Right.
2
SPEAKER_01
That s right.
2
SPEAKER_01
And if we refuse to talk about it,
2
SPEAKER_01
and then AGI gets built, who s going to be actually making the decision about whose values?
2
SPEAKER_01
It s going to be a bunch of dudes in some tech company, right?
2
SPEAKER_01
And are they necessarily?
2
SPEAKER_01
so representative of all of humankind that we want to just entrust it to them?
2
SPEAKER_01
Or are they even uniquely qualified to speak to future human happiness just because they re good at programming AI?
2
SPEAKER_01
I d much rather have this be a really inclusive conversation.
2
SPEAKER_00
But do you think it s possible, so you create a beautiful vision that includes the diversity, cultural diversity, and various perspectives on discussing rights, freedoms, human dignity.
2
SPEAKER_00
But how hard is it to come to that consensus?
2
SPEAKER_00
Do you think it s certainly a really important thing that we should all try to do, but do you think it s feasible?
2
SPEAKER_01
There s no better way to guarantee failure than to refuse to talk about it or refuse to try.
2
SPEAKER_01
I also think it s a really bad strategy to say, okay, let s first have a discussion for a long time and then once we reach complete consensus, then we ll try to load it into some machine.
2
SPEAKER_01
No, we shouldn t let perfect be the enemy of good.
2
SPEAKER_01
We should start with the kindergarten ethics that pretty much everybody agrees on and put that into machines now.
2
SPEAKER_01
We re not doing that even.
2
SPEAKER_01
Look at, you know, anyone who builds a passenger aircraft wants it to never, under any circumstances, fly into a building or a mountain, right?
2
SPEAKER_01
Yet the September 11th hijackers were able to do that.
2
SPEAKER_01
And even more embarrassingly, you know, Andreas Lubitz, this depressed German wings pilot, when he flew his passenger jet into the Alps, killing over a hundred people,
2
SPEAKER_01
He just told the autopilot to do it.
2
SPEAKER_01
He told the freaking computer to change the altitude to 100 meters.
2
SPEAKER_01
And even though it had the GPS maps, everything, the computer was like, okay.
2
SPEAKER_01
So we should take those very basic values where the problem is not that we don t agree.
2
SPEAKER_01
The problem is just we ve been too lazy to try to put it into our machines and make sure that from now on airplanes will just,
2
SPEAKER_01
which all have computers in them, but we just refuse to do something like that.
2
SPEAKER_01
Go into safe mode, maybe lock the cockpit door, go to the nearest airport.
2
SPEAKER_01
And there s so much other technology in our world as well now where it s really becoming quite timely to put in some sort of very basic values like this.
2
SPEAKER_01
Even in cars, we ve had enough vehicle terrorism attacks by now where people have driven trucks
2
SPEAKER_01
Advance into pedestrians that is not at all a crazy idea to just have that hardwired into the car.
2
SPEAKER_01
Yeah there are a lot of there s always gonna be people who for some reason wanna harm others but most of those people don t have the technical expertise to figure out how to work around something like that so if the car just won t do it.
2
SPEAKER_01
It helps.
2
SPEAKER_01
So let s start there.
2
SPEAKER_00
So there s a lot of, that s a, that s a great point.
2
SPEAKER_00
So not, not chasing perfect.
2
SPEAKER_00
There s a lot of things that a lot, that most of the world agrees on.
2
SPEAKER_00
Yeah.
2
SPEAKER_00
Let s start there.
2
SPEAKER_00
Let s start there.
2
SPEAKER_01
And then, and then once we start there, we ll also get into the habit of.
2
SPEAKER_01
having these kind of conversations about, okay, what else should we put in here and have these discussions.
2
SPEAKER_01
This should be a gradual process then.
2
SPEAKER_00
Great.
2
SPEAKER_00
So, but that also means describing these things and describing it to a machine.
2
SPEAKER_00
So one thing
2
SPEAKER_00
We had a few conversations with Stephen Wolfram.
2
SPEAKER_00
I m not sure if you re familiar with Stephen.
2
SPEAKER_01
Oh yeah, I know him quite well.
2
SPEAKER_00
So he works with a bunch of things, but cellular automata, these simple computable things, these computation systems.
2
SPEAKER_00
And he kind of mentioned that we probably have already, within these systems, already something that s AGI.
2
SPEAKER_00
Meaning like, we just don t know it because we can t talk to it.
2
SPEAKER_00
So if you give me this chance to try to at least form a question out of this is...
2
SPEAKER_00
I think it s an interesting idea to think that we can have intelligent systems, but we don t know how to describe something to them and they can t communicate with us.
2
SPEAKER_00
I know you re doing a little bit of work in explainable AI, trying to get AI to explain itself.
2
SPEAKER_00
So what are your thoughts of natural language processing or some kind of other communication?
2
SPEAKER_00
How does the AI explain something to us?
2
SPEAKER_00
How do we explain something to it, to machines?
2
SPEAKER_00
Or you think of it differently?
2
SPEAKER_01
So there are two separate,
2
SPEAKER_01
parts to your question there.
2
SPEAKER_01
One of them has to do with communication, which is super interesting, and we ll get to that in a sec.
2
SPEAKER_01
The other is whether we already have AGI or we just haven t noticed it.
2
SPEAKER_01
There I beg to differ.
2
SPEAKER_01
I don t think there s anything in any cellular automaton or the internet itself or whatever that has
2
SPEAKER_01
artificial general intelligence and that it can really do exactly everything we humans can do better.
2
SPEAKER_01
I think the day that happens, when that happens, we will very soon notice.
2
SPEAKER_01
We ll probably notice even before, because in a very, very big way.
2
SPEAKER_00
But for the second part though, you have this beautiful way of formulating consciousness as information processing, you can think of intelligence as information processing, you can think of the entire universe as these particles
2
SPEAKER_00
and these systems roaming around that have this information processing power.
2
SPEAKER_00
You don t think there is something with the power to process information in the way that we human beings do that s out there?
2
SPEAKER_00
that needs to be sort of connected to.
2
SPEAKER_00
It seems a little bit philosophical perhaps, but there s something compelling to the idea that the power is already there.
2
SPEAKER_00
The focus should be more on being able to communicate with it.
2
SPEAKER_01
Well, I agree that in a certain sense the hardware processing power is already out there because our universe itself
2
SPEAKER_01
You can think of it as being a computer already, right?
2
SPEAKER_01
It s constantly computing what water waves, how to evolve the water waves and the River Charles and how to move the air molecules around.
2
SPEAKER_01
Seth Lloyd has pointed out, my colleague here, that
2
SPEAKER_01
You can even in a very rigorous way think of our entire universe as being a quantum computer.
2
SPEAKER_01
It s pretty clear that our universe supports this amazing processing power because you can even, within this physics computer that we live in, we can even build actually laptops and stuff.
2
SPEAKER_01
So clearly the power is there.
2
SPEAKER_01
It s just that most of the compute power that nature has
2
SPEAKER_01
It s, in my opinion, kind of wasting on boring stuff, like simulating yet another ocean wave somewhere where no one is even looking.
2
SPEAKER_01
So in a sense, what life does, what we are doing when we build computers, is we re re-channeling all this compute that nature is doing anyway into doing things that are more interesting than just yet another ocean wave.
2
SPEAKER_01
Let s do something cool here.
2
SPEAKER_01
So the raw hardware power is there, for sure.
2
SPEAKER_01
And even just computing what s going to happen for the next five seconds in this water bottle, it takes a ridiculous amount of compute if you do it on a human computer.
2
SPEAKER_01
This water bottle just did it.
2
SPEAKER_01
But that does not mean that this water bottle has AGI.
2
SPEAKER_01
Because AGI means it should also be able to have written my book, done this interview.
2
SPEAKER_01
And I don t think it s just communication problems.
2
SPEAKER_01
I don t really know.
2
SPEAKER_01
It can do it.
2
SPEAKER_00
Although Buddhists say when they watch the water and that there is some beauty, that there is some depth and beauty in nature that they can communicate with.
2
SPEAKER_01
Communication is also very important because, I mean, look, part of my job is being a teacher and I know some very intelligent professors even who just
2
SPEAKER_01
have a better hard time communicating.
2
SPEAKER_01
They come up with all these brilliant ideas, but to communicate with somebody else, you have to also be able to simulate their own mind.
2
SPEAKER_01
Yes.
2
SPEAKER_00
Empathy.
2
SPEAKER_01
Build well enough a model of their mind that you can say things that they will understand.
2
SPEAKER_01
And that s quite difficult.
2
SPEAKER_01
And that s why today it s so frustrating if you have a computer that makes some cancer diagnosis and you ask it, well, why are you saying I should have this surgery?
2
SPEAKER_01
And if it can only reply, I was trained on five terabytes of data and this is my diagnosis, boop, boop, beep, beep.
2
SPEAKER_01
It doesn t really instill a lot of confidence, right?
2
SPEAKER_01
So I think we have a lot of work to do on that.
2
SPEAKER_00
on communication there.
2
SPEAKER_00
So what kind of, what kind of, I think you re doing a little bit work in explainable AI.
2
SPEAKER_00
What do you think are the most promising avenues?
2
SPEAKER_00
Is it mostly about sort of the Alexa problem of natural language processing of being able to actually use human interpretable methods of communication?
2
SPEAKER_00
So being able to talk to a system and talk back to you, or is there some more fundamental problems to be solved?
2
SPEAKER_01
I think it s all of the above.
2
SPEAKER_01
The natural language processing is obviously important, but there are also more nerdy fundamental problems.
2
SPEAKER_01
Like if you take... You play chess?
2
SPEAKER_00
Of course, I m Russian.
2
SPEAKER_00
I have to.
2
SPEAKER_00
Oh, you speak Russian?
2
SPEAKER_00
Yes, I speak Russian.
2
SPEAKER_01
Great, I didn t know.
2
SPEAKER_01
When did you learn Russian?
2
SPEAKER_01
I speak Russian very poorly.
2
SPEAKER_01
I m only an autodidact.
2
SPEAKER_01
I bought a book, Teach Yourself Russian.
2
SPEAKER_01
I read a lot, but it was very difficult.
2
SPEAKER_01
Wow.
2
SPEAKER_01
How many languages do you know?
2
SPEAKER_01
Wow, that s really impressive.
2
SPEAKER_01
I don t know.
2
SPEAKER_01
My wife has some calculations.
2
SPEAKER_01
But my point was, if you play chess, have you looked at the AlphaZero games?
2
SPEAKER_01
The actual games?
2
SPEAKER_01
No.
2
SPEAKER_01
Check it out.
2
SPEAKER_01
Some of them are just mind-blowing.
2
SPEAKER_01
Really beautiful.
2
SPEAKER_01
And if you ask, how did it do that?
2
SPEAKER_01
You go talk to Demis Hassabis and others from DeepMind, all they ll ultimately be able to give you is big tables of numbers, matrices that define the neural network.
2
SPEAKER_01
And you can stare at these numb tables of numbers until your face turned blue.
2
SPEAKER_01
And you re not going to understand much about why it made that move.
3
SPEAKER_00
And even if you have natural language processing that can tell you in human language about, oh, 5, 7, .28, still not going to really help.
3
SPEAKER_00
So I think there s a whole spectrum of fun challenges there involved in taking a computation that does intelligent things and transforming it into something equally good, equally intelligent.
3
SPEAKER_00
But that s more understandable.
3
SPEAKER_00
And I think that s really valuable because I think.
3
SPEAKER_00
As we put machines in charge of ever more infrastructure in our world, the power grid.
3
SPEAKER_00
Trading on the stock market weapon systems and so on, it s.
3
SPEAKER_00
Absolutely crucial that we can trust these guys to do what we want and trust really comes from understanding.
3
SPEAKER_00
right in a very fundamental way.
3
SPEAKER_00
And, um, that s why I m, that s why I m working on this.
3
SPEAKER_00
Cause I think the more, if we re going to have some hope of, of ensuring that machines have adopted our goals and that they re going to retain them, that kind of trust, I think needs to be based on things you can actually understand, preferably even make perfectly improved theorems on even with a self-driving car.
3
SPEAKER_00
Right.
3
SPEAKER_00
If someone just tells you it s been trained on tons of data and it never crashed, it s less reassuring than if someone actually has a proof.
3
SPEAKER_00
Maybe it s a computer-verified proof, but still, it says that under no circumstances is this car just going to swerve into oncoming traffic.
3
SPEAKER_01
And that kind of information helps build trust and helps build the alignment of goals, at least awareness that your goals, your values are aligned.
3
SPEAKER_00
And I think even in the very short term, if you look at how, you know, today, right, this absolutely pathetic state of cybersecurity that we have, where there s three billion Yahoo accounts, almost every American s credit card, and so on.
3
SPEAKER_00
Why is this happening?
3
SPEAKER_00
It s ultimately happening because we have software that nobody fully understood how it worked.
3
SPEAKER_00
That s why the bugs hadn t been found, right?
3
SPEAKER_00
And I think AI can be used very effectively for offense, for hacking, but it can also be used for defense, hopefully automating verifiability and creating systems that are built in different ways so you can actually prove things about them.
3
SPEAKER_00
And it s important.
3
SPEAKER_01
So speaking of software that nobody understands how it works, of course a bunch of people ask about your paper, about your thoughts of why does deep and cheap learning work so well.
3
SPEAKER_01
That s the paper, but what are your thoughts on deep learning?
3
SPEAKER_01
These kind of simplified models of our own brains have been able to do some successful perception work, pattern recognition work, and now with AlphaZero and so on, do some clever things.
3
SPEAKER_01
What are your thoughts about the promise, limitations of this piece?
3
SPEAKER_00
Great.
3
SPEAKER_00
I think there are a number of very important insights, very important lessons we can already draw from these kind of successes.
3
SPEAKER_00
One of them is when you look at the human brain, you see it s very complicated, 10th of 11 neurons, and there are all these different kinds of neurons and yada yada, and there s been this long debate about whether the fact that we have dozens of different kinds is actually necessary for intelligence.
3
SPEAKER_00
We can now, I think, quite convincingly answer that question with no.
3
SPEAKER_00
It s enough to have just one kind.
3
SPEAKER_00
If you look under the hood of alpha zero, there s only one kind of neuron, and it s ridiculously simple.
3
SPEAKER_00
It s a simple mathematical thing.
3
SPEAKER_00
It s just like in physics.
3
SPEAKER_00
If you have a gas with waves in it, it s not the detailed nature of the molecules that matter.
3
SPEAKER_00
It s the collective behavior somehow.
3
SPEAKER_00
Similarly, it s this higher level structure of the network that matters, not that you have 20 kinds of neurons.
3
SPEAKER_00
I think our brain is such a complicated mess because it wasn t evolved just to be intelligent.
3
SPEAKER_00
It was evolved to also be self-assembling and self-repairing.
3
SPEAKER_00
evolutionarily attainable.
3
SPEAKER_00
My hunch is that we re going to understand how to build AGI before we fully understand how our brains work.
3
SPEAKER_00
Just like we understood how to build flying machines long before we were able to build a mechanical bird.
3
SPEAKER_01
Yeah, that s right.
3
SPEAKER_01
You ve given the example exactly of mechanical birds and airplanes.
3
SPEAKER_01
And airplanes do a pretty good job of flying without really mimicking bird flight.
3
SPEAKER_00
And even now after 100 years later, did you see the TED talk with this German mechanical bird?
3
SPEAKER_00
I did not.
3
SPEAKER_00
I heard you mention it.
3
SPEAKER_00
Check it out.
3
SPEAKER_00
It s amazing.
3
SPEAKER_00
But even after that, we still don t fly mechanical birds because it turned out the way we came up with was simpler.
3
SPEAKER_00
And it s better for our purposes, and I think it might be the same there.
3
SPEAKER_00
That s one lesson.
3
SPEAKER_00
Another lesson, which is more what our paper was about.
3
SPEAKER_00
First, I, as a physicist, thought it was fascinating how there s a very close mathematical relationship, actually, between artificial neural networks and a lot of things that we ve studied for in physics that go by nerdy names like the renormalization group equation and Hamiltonians and yada, yada, yada.
3
SPEAKER_00
And when you look a little more closely at this, you have... At first, I was like, well, there s something crazy here that doesn t make sense.
3
SPEAKER_00
Because we know that if you even want to build a super simple neural network, tell apart cat pictures and dog pictures, right, that you can do that very, very well now.
3
SPEAKER_00
But if you think about it a little bit, you convince yourself it must be impossible, because if I have one megapixel, even if each pixel is just black or white, there s two to the power of one million possible images, which is way more than there are atoms in our universe.
3
SPEAKER_00
And then for each one of those, I have to assign a number, which is the probability that it s a dog.
3
SPEAKER_00
So an arbitrary function of images is a list of
3
SPEAKER_00
more numbers than there are atoms in our universe.
3
SPEAKER_00
So clearly I can t store that under the hood of my GPU or my computer.
3
SPEAKER_00
Yet somehow it works.
3
SPEAKER_00
So what does that mean?
3
SPEAKER_00
Well, it means that out of all of the problems that you could try to solve with a neural network, almost all of them are impossible to solve with a reasonably sized one.
3
SPEAKER_00
But then what we showed in our paper was that the fraction of all the problems that you could possibly pose, that we actually care about given the laws of physics, is also an infinitesimally tiny little part.
3
SPEAKER_00
And amazingly, they re basically the same part.
3
SPEAKER_01
Yeah, it s almost like our world was created for us.
3
SPEAKER_01
I mean, they kind of come together.
3
SPEAKER_00
Yeah, but you could say maybe the world was created for us, but I have a more modest interpretation, which is that instead evolution endowed us with neural networks, precisely for that reason.
3
SPEAKER_00
Because this particular architecture, as opposed to the one on your laptop, is very, very well
3
SPEAKER_00
adapted to solving the kind of problems that nature kept presenting our ancestors with, right?
3
SPEAKER_00
So it makes sense that why do we have a brain in the first place?
3
SPEAKER_00
It s to be able to make predictions about the future and so on.
3
SPEAKER_00
So if we had a sucky system, which could never solve it, wouldn t have evolved.
3
SPEAKER_00
So, but it s, so this is, this is, uh, I think it s a very beautiful fact.
3
SPEAKER_00
Yeah.
3
SPEAKER_00
We also, we also realize that there s that there, that we, there ve been, it s been earlier work on, on
3
SPEAKER_00
why deeper networks are good, but we were able to show an additional cool fact there, which is that even incredibly simple problems, like suppose I give you a thousand numbers and ask you to multiply them together, you can write a few lines of code, boom, done, trivial.
3
SPEAKER_00
If you just try to do that with a neural network that has only one single hidden layer in it, you can do it.
3
SPEAKER_00
but you re going to need two to the power of a thousand neurons to multiply a thousand numbers, which is again more neurons than there are atoms in our universe.
3
SPEAKER_00
That s fascinating.
3
SPEAKER_00
But if you allow yourself to make it a deep network of many layers, you only need 4,000 neurons.
3
SPEAKER_00
It s perfectly feasible.
3
SPEAKER_01
That s really interesting.
3
SPEAKER_01
Yeah.
3
SPEAKER_01
So on another architecture type, I mean, you mentioned Schrodinger s equation and what are your thoughts about quantum computing and the role of this kind of computational unit in creating an intelligent system?
3
SPEAKER_00
In some Hollywood movies?
3
SPEAKER_00
I will not mention my name because I don t want to spoil them.
3
SPEAKER_00
The way they get AGI is building a quantum computer.
3
SPEAKER_00
Because the word quantum sounds cool and so on.
3
SPEAKER_00
First of all, I think we don t need quantum computers to build AGI.
3
SPEAKER_00
I suspect your brain is not a quantum computer in any profound sense.
3
SPEAKER_00
I even wrote a paper about that many years ago.
3
SPEAKER_00
I calculated the so-called decoherence time, how long it takes until the quantum computerness of what your neurons are doing gets erased by just random noise from the environment.
3
SPEAKER_00
And it s about 10 to the minus 21 seconds.
3
SPEAKER_00
So as cool as it would be to have a quantum computer in my head, I don t think that fast.
3
SPEAKER_00
On the other hand, there are
3
SPEAKER_00
very cool things you could do with quantum computers, or I think we ll be able to do soon when we get bigger ones, that might actually help machine learning do even better than the brain.
3
SPEAKER_00
For example, this is just a moonshot, but learning
3
SPEAKER_00
It s very much the same thing as search.
3
SPEAKER_00
If you re trying to train a neural network to get really learned, to do something really well, you have some loss function, you have a bunch of knobs you can turn, represented by a bunch of numbers, and you re trying to tweak them so that it becomes as good as possible at this thing.
3
SPEAKER_00
So if you think of a landscape with some valley,
3
SPEAKER_00
where each dimension of the landscape corresponds to some number you can change, you re trying to find the minimum.
3
SPEAKER_00
And it s well known that if you have a very high dimensional landscape, complicated things, it s super hard to find the minimum, right?
3
SPEAKER_00
Quantum mechanics is amazingly good at this.
3
SPEAKER_00
If I want to know what s the lowest energy state this water can possibly have, incredibly hard to compute, but nature will happily figure this out for you if you just cool it down, make it very, very cold.
3
SPEAKER_00
If you put a ball somewhere, it ll roll down to its minimum.
3
SPEAKER_00
And this happens metaphorically at the energy landscape, too.
3
SPEAKER_00
And quantum mechanics even uses some clever tricks, which today s machine learning systems don t.
3
SPEAKER_00
If you re trying to find the minimum and you get stuck in the little local minimum here, in quantum mechanics, you can actually tunnel through the barrier and get unstuck again.
3
SPEAKER_00
That s really interesting.
3
SPEAKER_00
So maybe, for example, we ll one day use quantum computers that
3
SPEAKER_00
to help train neural networks better.
3
SPEAKER_01
That s really interesting.
3
SPEAKER_01
Okay, so as a component of kind of the learning process, for example.
3
SPEAKER_00
Yeah.
3
SPEAKER_01
Let me ask, sort of wrapping up here a little bit, let me return to the questions of our human nature and love, as I mentioned.
3
SPEAKER_01
So, do you think, you mentioned sort of a helper robot, but you can think of also personal robots.
3
SPEAKER_01
Do you think the way we human beings fall in love and get connected to each other is possible to achieve in an AI system, a human level AI intelligence system?
3
SPEAKER_01
Do you think we would ever see that kind of connection?
3
SPEAKER_01
You know, in all this discussion about solving complex goals, is this kind of human social connection, do you think that s one of the goals on the peaks and valleys with the raising sea levels that we ll be able to achieve?
3
SPEAKER_01
Or do you think that s something that s ultimately, or at least in the short term, relative to the other goals, is not achievable?
3
SPEAKER_00
I think it s all possible.
3
SPEAKER_00
I mean, in recent, there s a very wide range of guesses, as you know, among AI researchers, when we re going to get AGI.
3
SPEAKER_00
Some people, like our friend Rodney Brooks, said it s going to be hundreds of years at least.
3
SPEAKER_00
And then there are many others who think it s going to happen much sooner.
3
SPEAKER_00
And recent polls, maybe half or so of AI researchers think we re going to get AGI within decades.
3
SPEAKER_00
So if that happens, of course, then I think these things are all possible.
3
SPEAKER_00
But in terms of whether it will happen,
3
SPEAKER_00
I think we shouldn t spend so much time asking, what do we think will happen in the future?
3
SPEAKER_00
As if we are just some sort of pathetic, passive bystanders, waiting for the future to happen to us.
3
SPEAKER_00
Hey, we re the ones creating this future, right?
3
SPEAKER_00
So we should be proactive about it and ask ourselves what sort of future we would like to have happen.
3
SPEAKER_00
That s right.
3
SPEAKER_00
Trying to make it like that.
3
SPEAKER_00
Well, would I prefer it to some sort of incredibly boring zombie-like future where there s all these mechanical things happening and there s no passion, no emotion, no experience maybe even?
3
SPEAKER_00
No, I would of course much rather prefer it if all the things that we find that we value the most about humanity
3
SPEAKER_00
our subjective experience, passion, inspiration, love.
3
SPEAKER_00
If we can create a future where those things do exist, I think ultimately it s not our universe giving meaning to us, it s us giving meaning to our universe.
3
SPEAKER_00
And if we build more advanced intelligence, let s make sure we build it in such a way that meaning is
3
SPEAKER_00
is part of it.
3
SPEAKER_01
A lot of people that seriously study this problem and think of it from different angles have trouble in the majority of cases if they think through that happen are the ones that are not beneficial to humanity.
3
SPEAKER_01
So what are your thoughts, what should people
3
SPEAKER_01
you know, I really don t like people to be terrified.
3
SPEAKER_01
What s a way for people to think about it in a way that, in a way we can solve it and we can make it better?
3
SPEAKER_00
Yeah.
3
SPEAKER_00
No, I don t think panicking is going to help in any way.
3
SPEAKER_00
It s not going to increase chances of things going well either.
3
SPEAKER_00
Even if you are in a situation where there is a real threat, does it help if everybody just freaks out?
3
SPEAKER_00
Right.
3
SPEAKER_00
No, of course, of course not.
3
SPEAKER_00
I think, yeah, there are, of course, ways in which things can go horribly wrong.
3
SPEAKER_00
First of all, it s important when we think about this thing, about the problems and risks, to also remember how huge the upsides can be if we get it right.
3
SPEAKER_00
Everything we love about society and civilization is a product of intelligence.
3
SPEAKER_00
So if we can amplify our intelligence with machine intelligence and not anymore lose our loved ones to what we re told is an uncurable disease and things like this, of course
3
SPEAKER_00
We should aspire to that.
3
SPEAKER_00
So that can be a motivator, I think, reminding ourselves that the reason we try to solve problems is not just because we re trying to avoid gloom, but because we re trying to do something great.
3
SPEAKER_00
But then in terms of the risks, I think
3
SPEAKER_00
The really important question is to ask, what can we do today that will actually help make the outcome good?
3
SPEAKER_00
Dismissing the risk is not one of them.
3
SPEAKER_00
I find it quite funny often when I m on discussion panels about these things, how the people who
3
SPEAKER_00
who work for companies always say, oh, nothing to worry about, nothing to worry about, nothing to worry about.
3
SPEAKER_00
And it s only academics sometimes express concerns.
3
SPEAKER_00
That s not surprising at all if you think about it.
3
SPEAKER_00
Upton Sinclair quipped that it s hard to make a man believe in something when his income depends on not believing in it.
3
SPEAKER_00
Frankly, we know a lot of these people in companies that they re just as concerned as anyone else.
3
SPEAKER_00
But if you re the CEO of a company, that s not something you want to go on record saying when you have silly journalists who are going to put a picture of a Terminator robot when they quote you.
3
SPEAKER_00
So the issues are real.
3
SPEAKER_00
And the way I think about what the issue is, is basically the real choice we have is
3
SPEAKER_00
First of all, are we going to just dismiss this, the risks, and say, well, let s just go ahead and build machines that can do everything we can do better and cheaper.
3
SPEAKER_00
Let s just make ourselves obsolete as fast as possible.
3
SPEAKER_00
What could possibly go wrong?
3
SPEAKER_00
That s one attitude.
3
SPEAKER_00
The opposite attitude, I think, is to say, there s this incredible potential.
3
SPEAKER_00
Let s think about what kind of future we re really, really excited about.
3
SPEAKER_00
What are the shared goals?
3
SPEAKER_00
that we can really aspire towards.
3
SPEAKER_00
And then let s think really hard about how we can actually get there.
3
SPEAKER_00
Don t start thinking about the risks, start thinking about the goals.
3
SPEAKER_00
And then when you do that, then you can think about the obstacles you want to avoid.
3
SPEAKER_00
I often get students coming in right here into my office for career advice.
3
SPEAKER_00
I always ask them this very question, where do you want to be in the future?
3
SPEAKER_00
If all she can say is, oh, maybe I ll have cancer, maybe I ll run over by a truck.
3
SPEAKER_00
She s just going to end up a hypochondriac paranoid.
3
SPEAKER_00
Whereas if she comes in and fire in her eyes and is like, I want to be there.
3
SPEAKER_00
And then we can talk about the obstacles and see how we can circumvent them.
3
SPEAKER_00
That s, I think, a much, much healthier attitude.
3
SPEAKER_00
That s really well put.
3
SPEAKER_00
And I feel it s very challenging to come up with a vision for the future, which we are unequivocally excited about.
3
SPEAKER_00
I m not just talking now in the vague terms like, yeah, let s cure cancer.
3
SPEAKER_00
Fine.
3
SPEAKER_00
I m talking about what kind of society do we want to create?
3
SPEAKER_00
What do we want it to mean to be human in the age of AI, in the age of AGI?
3
SPEAKER_00
So if we can have this conversation,
4
SPEAKER_01
broad, inclusive conversation, and gradually start converging towards some future with some direction, at least, that we want to steer towards, then we ll be much more motivated to constructively take on the obstacles.
4
SPEAKER_01
And I think if I try to wrap this up in a more succinct way, I think we can all agree already now that we should aspire
4
SPEAKER_01
to build AGI that doesn t overpower us, but that empowers us.
4
SPEAKER_00
And think of the many various ways they can do that, whether that s from my side of the world of autonomous vehicles.
4
SPEAKER_00
I m personally actually from the camp that believes this human level intelligence is required to achieve something like vehicles that would actually be something we would enjoy using and being part of.
4
SPEAKER_00
So that s one example and certainly there s a lot of other types of robots and medicine and so on.
4
SPEAKER_00
So focusing on those and then coming up with the obstacles, coming up with the ways that that can go wrong and solving those one at a time.
4
SPEAKER_01
And just because you can build an autonomous vehicle, even if you could build one that would drive just fine without you, maybe there are some things in life that we would actually want to do ourselves.
4
SPEAKER_01
That s right.
4
SPEAKER_01
Like for example,
4
SPEAKER_01
if you think of our society as a whole, there s some things that we find very meaningful to do.
4
SPEAKER_01
And that doesn t mean we have to stop doing them just because machines can do them better.
4
SPEAKER_01
I m not going to stop playing tennis just the day someone builds a tennis robot and beat me.
4
SPEAKER_00
People are still playing chess and even go.
4
SPEAKER_01
Yeah.
4
SPEAKER_01
And in the very near term even, some people are advocating basic income, replace jobs.
4
SPEAKER_01
But if the government is going to be willing to just hand out cash to people for doing nothing,
4
SPEAKER_01
then one should also seriously consider whether the government should also just hire a lot more teachers and nurses and the kind of jobs which people often find great fulfillment in doing, right?
4
SPEAKER_01
I get very tired of hearing politicians saying, oh, we can t afford hiring more teachers, but we re going to maybe have basic income.
4
SPEAKER_01
If we can have more serious research and thought into what gives meaning to our lives, the jobs give so much more than income, right?
4
SPEAKER_01
And then think about in the future, what are the roles that we want to have?
4
SPEAKER_01
are people continually feeling empowered by machines.
4
SPEAKER_00
And I think, sort of, I come from Russia, from the Soviet Union, and I think for a lot of people in the 20th century, going to the moon, going to space was an inspiring thing.
4
SPEAKER_00
I feel like the universe of the mind, so AI, understanding, creating intelligence, is that for the 21st century.
4
SPEAKER_00
So it s really surprising, and I ve heard you mention this, it s really surprising to me, both on the research funding side, that it s not funded as greatly as it could be, but most importantly on the politician side, that it s not part of the public discourse except in the killer bots, Terminator kind of view, that people are not yet, I think, perhaps excited by the possible positive future that we can build together.
4
SPEAKER_01
And we should be, because
4
SPEAKER_01
Politicians usually just focus on the next election cycle, right?
4
SPEAKER_01
The single most important thing I feel we humans have learned in the entire history of science is they were the masters of underestimation.
4
SPEAKER_01
We underestimated the size of our cosmos again and again, realizing that everything we thought existed was just a small part of something grander, right?
4
SPEAKER_01
Planet, solar system, a galaxy, you know, clusters of galaxies, universe.
4
SPEAKER_01
And we now know
4
SPEAKER_01
that the future has just so much more potential than our ancestors could ever have dreamt of.
4
SPEAKER_01
This cosmos, imagine if all of Earth was completely devoid of life except for Cambridge, Massachusetts.
4
SPEAKER_01
Wouldn t it be kind of lame if all we ever aspired to was to stay in Cambridge, Massachusetts forever?
4
SPEAKER_01
and then go extinct in one week, even though Earth was gonna continue on for longer.
4
SPEAKER_01
That sort of attitude, I think, we have now on the cosmic scale.
4
SPEAKER_01
Life can flourish on Earth, not for four years, but for billions of years.
4
SPEAKER_01
I can even tell you about how to move it out of harm s way when the sun gets too hot.
4
SPEAKER_01
And then we have so much more resources out here, which today, maybe there are a lot of other planets with bacteria or cow-like life on them,
4
SPEAKER_01
Most of this, all this opportunity seems, as far as we can tell, to be largely dead, like the Sahara Desert, and yet we have the opportunity to help life flourish around this for billions of years.
4
SPEAKER_01
So let s quit squabbling about whether some little border should be drawn one mile to the left or right.
4
SPEAKER_01
look up into the skies and realize, hey, you know, we can do such incredible things.
4
SPEAKER_00
Yeah.
4
SPEAKER_00
And that s, I think, why it s really exciting that you and others are connected with some of the work Elon Musk is doing, because he s literally going out into that space, really exploring our universe.
4
SPEAKER_00
And it s wonderful.
4
SPEAKER_01
That is exactly why Elon Musk is so misunderstood, right?
4
SPEAKER_01
Misconstrued him as some kind of pessimistic doomsayer.
4
SPEAKER_01
The reason he cares so much about AI safety is because he more than
4
SPEAKER_01
Almost anyone else appreciates these amazing opportunities that will squander if we wipe out here on Earth.
4
SPEAKER_01
We re not just going to wipe out the next generation, but all generations.
4
SPEAKER_01
And this incredible opportunity that s out there, and that would really be a waste.
4
SPEAKER_01
And AI, for people who think that it would be better to do without technology, let me just mention that
4
SPEAKER_01
If we don t improve our technology, the question isn t whether humanity is going to go extinct.
4
SPEAKER_01
The question is just whether we re going to get taken out by the next big asteroid or the next super volcano or something else dumb that we could easily prevent with more tech, right?
4
SPEAKER_01
And if we want life to flourish throughout the cosmos, AI is the key to it.
4
SPEAKER_01
As I mentioned in a lot of detail in my book right there, even many of the most
4
SPEAKER_01
Inspired sci-fi writers, I feel, have totally underestimated the opportunities for space travel, especially to other galaxies, because they weren t thinking about the possibility of AGI, which just makes it so much easier.
4
SPEAKER_00
Right.
4
SPEAKER_00
Yeah.
4
SPEAKER_00
So that goes to your view of AGI that enables our progress, that enables a better life.
4
SPEAKER_00
So that s a beautiful way to put it and something to strive for.
4
SPEAKER_00
So Max, thank you so much.
4
SPEAKER_00
Thank you for your time today.
4
SPEAKER_00
It s been awesome.
4
SPEAKER_01
Thank you so much.