0
SPEAKER_00
The following is a conversation with Wojciech Zaremba, co-founder of OpenAI, which is one of the top organizations in the world doing artificial intelligence research and development.
0
SPEAKER_00
Wojciech is the head of language and co-generation teams.
0
SPEAKER_00
building and doing research on GitHub Copilot, OpenAI Codecs, and GPT-3, and who knows, 4, 5, 6, N, and N plus one.
0
SPEAKER_00
And he also previously led OpenAI s robotic efforts.
0
SPEAKER_00
These are incredibly exciting projects to me that deeply challenge and expand our understanding of the structure and nature of intelligence.
0
SPEAKER_00
The 21st century, I think, may very well be remembered for a handful of revolutionary AI systems and their implementations.
0
SPEAKER_00
GPT, codecs, and applications of language models and transformers in general to the language and visual domains may very well be at the core of these AI systems.
0
SPEAKER_00
To support this podcast, please check out our sponsors.
0
SPEAKER_00
They re listed in the description.
0
SPEAKER_00
As usual, I ll do a few minutes of ads now, no ads in the middle.
0
SPEAKER_00
I try to make these interesting, so hopefully you don t skip, but if you do, please still check out the sponsor links in the description.
0
SPEAKER_00
It s the best way to support this podcast.
0
SPEAKER_00
I use their stuff.
0
SPEAKER_00
I enjoy it.
0
SPEAKER_00
Maybe you will too.
0
SPEAKER_00
This show is brought to you by Paperspace Gradient, which is a platform that lets you build, train, deploy machine learning models of any size and complexity.
0
SPEAKER_00
I love how powerful and intuitive Gradient is.
0
SPEAKER_00
If you re already in the machine learning world or are interested in joining it, I highly recommend Fast.ai, which is a set of courses by Jeremy Howard and others that introduce you to machine learning.
0
SPEAKER_00
And the reason I bring that up is they use and recommend Paper Space Gradient.
0
SPEAKER_00
One of the nice things, you can host notebooks on there, you can swap out the compute instance at any time, so you can start out with a small-scale CPU, GPU instance, and then swap out something super powerful once you re ready, once you need it.
0
SPEAKER_00
I m also really excited about what they re calling workflows, which provides a way to automate machine learning pipelines on top of gradient compute infrastructure.
0
SPEAKER_00
It makes it really easy to build a production app because all the orchestration is reduced to a simple YAML file.
0
SPEAKER_00
To give Gradient a try, visit gradient.run slash lex and use the signup link there.
0
SPEAKER_00
You ll get 15 bucks in free credits which you can use to power your next machine learning application.
0
SPEAKER_00
That s gradient.run slash lex.
0
SPEAKER_00
If you re interested in future machine learning projects I m doing in terms of tutorials and educational stuff,
0
SPEAKER_00
Maybe Gradient is worth a try because I m likely to use Gradient as part of that.
0
SPEAKER_00
Anyway, like I said, gradient.run slash Lex.
0
SPEAKER_00
This episode is also brought to you by Indeed, a hiring website.
0
SPEAKER_00
I ve used them as part of many hiring efforts I ve done for the teams I ve led in the past.
0
SPEAKER_00
They have tools like Indeed Instant Match, giving you quality candidates whose resumes on Indeed fit your job description immediately.
0
SPEAKER_00
I think hiring is the most important component of a successful company, no matter what stage your company is in.
0
SPEAKER_00
I ve been lucky to get a chance to interact with some very successful people in the entrepreneurial and the business world.
0
SPEAKER_00
And I think that universally the thing that is common between them is the ability to build a great team.
0
SPEAKER_00
So you should definitely use the best tools for the job and indeed should be in your toolbox.
0
SPEAKER_00
Right now, get a free $75 sponsored job credit to upgrade your job post at indeed.com slash Lex.
0
SPEAKER_00
Get it at indeed.com slash Lex.
0
SPEAKER_00
Offer valid through September 30th.
0
SPEAKER_00
Terms and conditions apply.
0
SPEAKER_00
Join three million businesses that use Indeed by going to indeed.com slash Lex.
0
SPEAKER_00
This episode is also brought to you by Blinkist, my favorite app for learning new things.
0
SPEAKER_00
Blinkist takes the key ideas from thousands of nonfiction books and condenses them down into just 15 minutes that you can read or listen to.
0
SPEAKER_00
I can recommend a bunch of books, including Sapiens by Yuval Harari.
0
SPEAKER_00
I recently announced that I m doing a book a day for a few days and actually had some chaos come up in my life, which threw a wrench into that.
0
SPEAKER_00
but I will return to this project and I m excited by the idea of sort of spending an entirety of a day reading and really sort of living through the story.
0
SPEAKER_00
But I use Blinkist to select possible nonfiction books and I also use Blinkist to reflect on the books I ve read in the past whether I want to revisit them.
0
SPEAKER_00
Go to blinkist.com slash Lex to start your free seven day trial and get 25% off a Blinkist premium membership.
0
SPEAKER_00
That s blinkist.com slash Lex spelled
0
SPEAKER_00
B-L-I-N-K-I-S-T.
0
SPEAKER_00
Blinkist.com slash Lex.
0
SPEAKER_00
This show is also brought to you by Grammarly, a writing assistant tool that checks spelling, grammar, sentence structure, and readability.
0
SPEAKER_00
Grammarly Premium, the version you pay for, offers a bunch of extra features.
0
SPEAKER_00
My favorite is the clarity check, which helps detect rambling, overcomplicated chaos that many of us can descend into.
0
SPEAKER_00
I m a big fan of quality of writing emerging through the editing process.
0
SPEAKER_00
So the stream of consciousness where you just throw a bunch of stuff on the page and figure out which of it sticks.
0
SPEAKER_00
For me, the end result of a great piece of writing is as simple as possible.
0
SPEAKER_00
And like Einstein said, not too simple.
0
SPEAKER_00
But I find it s almost difficult to arrive at something that s too simple.
0
SPEAKER_00
Grammarly is available on basically any platform and major sites and apps like Gmail and Twitter and so on.
0
SPEAKER_00
Do more than just spellcheck.
0
SPEAKER_00
Get your point across more effectively with Grammarly Premium.
0
SPEAKER_00
Get 20% off by signing up at grammarly.com slash lex.
0
SPEAKER_00
That s grammarly.com slash lex.
0
SPEAKER_00
This episode is also brought to you by Eight Sleep and it s Pod Pro mattress.
0
SPEAKER_00
It controls temperature with an app, it s like magic.
0
SPEAKER_00
It s back with sensors, it can cool down to as low as 55 degrees on each side of the bed separately.
0
SPEAKER_00
I ve been hanging out with Andrew Huberman this week and he s actually just a mastermind of temperature control on all kinds of devices.
0
SPEAKER_00
He had this nice explanation of what parts of the body to apply cooling to for it to be actually effective.
0
SPEAKER_00
It s fascinating to see what kind of effect temperature control has on you.
0
SPEAKER_00
I think during sleep, for me, it does magic.
0
SPEAKER_00
Even with air conditioning, just having a warm blanket and a cold bed makes me really look forward to the naps and the sleep.
0
SPEAKER_00
They have a Pod Pro cover so you can just add that to your mattress without having to buy theirs.
0
SPEAKER_00
But their mattress is nice, just so you know.
0
SPEAKER_00
Again, it can track a bunch of metrics like heart rate variability, but cooling alone is honestly worth the money.
0
SPEAKER_00
Go to 8sleep.com slash Lex to get special savings.
0
SPEAKER_00
That s 8sleep.com slash Lex.
0
SPEAKER_00
This is the Lex Friedman Podcast, and here s my conversation with Wojciech Saremba.
0
SPEAKER_00
You mentioned that Sam Altman asked about the Fermi Paradox, and the people at OpenAI had really sophisticated, interesting answers.
0
SPEAKER_00
So that s when you knew this is the right team to be working with.
0
SPEAKER_00
So let me ask you about the Fermi Paradox, about aliens.
0
SPEAKER_00
Why have we not found overwhelming evidence for aliens visiting Earth?
0
SPEAKER_01
I don t have a conviction in the answer, but rather a kind of probabilistic perspective on what might be a, let s say, possible answers.
0
SPEAKER_01
It s also interesting that the question itself even can touch on the, you know, your typical question of what s the meaning of life?
0
SPEAKER_01
Because if you assume that, like, we don t see aliens because they destroy themselves, that kind of upweights the focus on making sure that we won t destroy ourselves.
0
SPEAKER_01
And at the moment,
0
SPEAKER_01
place where I am actually with my belief and these things also change over the time is I think that we might be alone in the universe which actually makes life more or let s say consciousness life more kind of valuable and that means that we should more appreciate it
0
SPEAKER_00
Have we always been alone?
0
SPEAKER_00
So what s your intuition about our galaxy, our universe?
0
SPEAKER_00
Is it just sprinkled with graveyards of intelligent civilizations or are we truly, is life, intelligent life truly unique?
0
SPEAKER_01
At the moment, my belief that it is unique, but I would say I could also, you know, there was like some footage released with UFO objects, which makes me actually doubt my own belief.
0
SPEAKER_01
Yes.
0
SPEAKER_01
Yeah, I can tell you one crazy answer that I have heard.
0
SPEAKER_01
Yes.
0
SPEAKER_01
So apparently, when you look actually at the limits of computation, you can compute more if the temperature of the universe would drop down.
0
SPEAKER_01
So one of the things that aliens might want to do if they are truly optimizing to maximize amount of compute, which, you know, maybe can lead to more, let s say, simulations or so, it s instead of wasting current entropy of the universe, because, you know, we, by living, we are actually somewhat wasting entropy, then you can wait for the universe to cool down such that you have more computation.
0
SPEAKER_01
So that s kind of a funny answer.
0
SPEAKER_01
I m not sure if I believe in it.
0
SPEAKER_01
But that would be one of the reasons why you don t see aliens.
0
SPEAKER_01
It s also possible to some people say that maybe there is not that much point in actually going to other galaxies if you can go inwards.
0
SPEAKER_01
So there is no limits of what could be an experience if we could, you know, connect machines to our brains.
0
SPEAKER_01
While there are still some limits if you want to explore universe.
0
SPEAKER_00
Yeah, there could be a lot of ways to go inwards to once you figure out some aspect of physics we haven t figured out yet.
0
SPEAKER_00
Maybe you can travel to different dimensions.
0
SPEAKER_00
I mean.
0
SPEAKER_00
Travel in three dimensional space may not be the most fun kind of travel.
0
SPEAKER_00
There may be like just a huge amount of different ways to travel, and it doesn t require a spaceship going slowly in 3D space to space time.
0
SPEAKER_01
It also feels, you know, one of the problems is that speed of light is low and the universe is vast.
0
SPEAKER_01
And it seems that actually, most likely, if we want to travel very far, then we would, instead of actually sending spaceships with humans that weigh a lot, we would
0
SPEAKER_01
send something similar to what Yuri Miller is working on.
0
SPEAKER_01
These are like a huge sail, which is at first powered.
0
SPEAKER_01
There is a shot of laser from an air and it can propel it to quarter of speed of light and sail itself contains a few grams of equipment and that might be the way to actually
0
SPEAKER_01
transport matter through universe.
0
SPEAKER_01
But then when you think what would it mean for humans, it means that we would need to actually put their 3D printer and, you know, 3D print a human on other planet.
0
SPEAKER_01
I don t know, play them YouTube or let s say, or like a 3D print a huge human right away or maybe a womb or so.
0
SPEAKER_01
Yeah.
0
SPEAKER_00
With our current techniques of archaeology,
0
SPEAKER_00
If a civilization was born and died long, long enough ago on Earth, we wouldn t be able to tell.
0
SPEAKER_00
And so that makes me really sad.
0
SPEAKER_00
And so I think about Earth in that same way.
0
SPEAKER_00
How can we leave some remnants if we do destroy ourselves?
0
SPEAKER_00
How can we leave remnants for aliens in the future to discover?
0
SPEAKER_00
Like, here s some nice stuff we ve done.
0
SPEAKER_00
like Wikipedia and YouTube?
0
SPEAKER_00
Do we have it like in a satellite orbiting Earth with a hard drive?
0
SPEAKER_00
Like how do we say, how do we back up human civilization for the good parts or all of it is good parts?
0
SPEAKER_00
so that it can be preserved longer than our bodies can.
0
SPEAKER_00
It s a difficult question.
0
SPEAKER_00
It also requires the difficult acceptance of the fact that we may die.
0
SPEAKER_00
And if we die, we may die suddenly as a civilization.
0
SPEAKER_01
So let s see.
0
SPEAKER_01
I think it kind of depends on the cataclysm.
0
SPEAKER_01
We have observed in other parts of the universe that births of gamma rays.
0
SPEAKER_01
These are high energy rays of light that actually can apparently kill an entire galaxy.
0
SPEAKER_01
So there might be actually nothing even to, nothing to protect us from it.
0
SPEAKER_01
I m also, when I m looking actually at the past civilizations, so it s like Aztecs or so, they disappeared from the surface of the earth and one can ask, why is it the case?
0
SPEAKER_01
The way I m thinking about it is, you know, that definitely they had some problem that they couldn t solve.
0
SPEAKER_01
And maybe there was a flood and all of a sudden they couldn t drink, there was no potable water and they all died.
0
SPEAKER_01
I think that so far, the best solution to such problems is, I guess, technology.
0
SPEAKER_01
So, I mean, if they would know that you can just boil water and then drink it after, then that would save their civilization.
0
SPEAKER_01
And even now, when we look actually at the current pandemic, it seems that once again, actually, science comes to rescue.
0
SPEAKER_01
And somehow science increases size of the action space.
0
SPEAKER_01
And I think that s a good thing.
0
SPEAKER_00
Yeah, but nature has a vastly larger action space.
0
SPEAKER_01
But still, it might be a good thing for us to keep on increasing action space.
0
SPEAKER_00
OK.
0
SPEAKER_00
Looking at past civilizations, yes.
0
SPEAKER_00
But looking at the destruction of human civilization,
0
SPEAKER_00
perhaps expanding the action space will add actions that are easily acted upon, easily executed, and as a result, destroy us.
0
SPEAKER_01
So let s see.
0
SPEAKER_01
I was pondering why actually even we have negative impact on the globe.
0
SPEAKER_01
Because if you ask every single individual, they would like to have clean air.
0
SPEAKER_01
They would like healthy planet, but somehow it actually is not the case that as a collective, we are not going in this direction.
0
SPEAKER_01
I think that there exists very powerful system to describe what we value.
0
SPEAKER_01
That s capitalism.
0
SPEAKER_01
It assigns actually monetary values to various activities.
0
SPEAKER_01
At the moment, the problem in the current system is that there are some things which we value.
0
SPEAKER_01
There is no cost assigned to it.
0
SPEAKER_01
So even though we value clean air or maybe we also
0
SPEAKER_01
Value lack of destruction on the internet or so at the moment.
0
SPEAKER_01
These quantities you know companies corporations can pollute them for free.
0
SPEAKER_01
So in some sense.
0
SPEAKER_01
I wish, or like that s I guess purpose of politics, to align the incentive systems.
0
SPEAKER_01
And we are kind of maybe even moving in this direction.
0
SPEAKER_01
The first issue is even to be able to measure the things that we value.
0
SPEAKER_01
Then we can actually assign the monetary value to them.
0
SPEAKER_00
Yeah, and that s so it s getting the data and also probably through technology, enabling people to vote and to move money around in a way that is aligned with their values.
0
SPEAKER_00
And that s very much a technology question.
0
SPEAKER_00
So like having one president and Congress and voting that happens every four years or something like that, that s a very outdated idea.
0
SPEAKER_00
That could be some technological improvements to that kind of idea.
0
SPEAKER_01
So I m thinking from time to time about these topics, but it also feels to me that it s a little bit like it s hard for me to actually make correct predictions what is the appropriate thing to do.
0
SPEAKER_01
I extremely trust Sam Altman, our CEO, on these topics.
0
SPEAKER_01
I m more on the side of being, I guess, naive hippie.
0
SPEAKER_00
That s your life philosophy.
0
SPEAKER_00
Well, I think self doubt.
0
SPEAKER_00
And I think hippie implies optimism.
0
SPEAKER_00
Those those two things are pretty, pretty good way to operate.
0
SPEAKER_01
I mean, still, it is hard for me to actually understand how the politics works, or like, how this like, exactly how the things would play out.
0
SPEAKER_01
And Sam is really excellent with it.
0
SPEAKER_00
What do you think is rarest in the universe?
0
SPEAKER_00
You said we might be alone.
0
SPEAKER_00
What s hardest to build is another engineering way to ask that.
0
SPEAKER_00
Life, intelligence or consciousness?
0
SPEAKER_00
So like you said that we might be alone.
0
SPEAKER_00
Which is the thing that s hardest to get to?
0
SPEAKER_00
Is it just the origin of life?
0
SPEAKER_00
Is it the origin of intelligence?
0
SPEAKER_00
Is it the origin of consciousness?
0
SPEAKER_01
So let me at first explain you my kind of mental model, what I think is needed for life to appear.
0
SPEAKER_01
So I imagine that at some point there was this primordial soup of amino acids and maybe some proteins in the ocean.
0
SPEAKER_01
And some proteins were turning into some other proteins through reaction.
0
SPEAKER_01
And you can almost think about this cycle of what turns into what.
0
SPEAKER_01
There is a graph essentially describing which substance turns into some other substance.
0
SPEAKER_01
And essentially life means that all of a sudden in the graph has been created a cycle such that the same thing keeps on happening over and over again.
0
SPEAKER_01
That s what is needed for life to happen.
0
SPEAKER_01
And in some sense, you can think almost that you have this gigantic graph and it needs like a sufficient number of edges for the cycle
1
SPEAKER_02
to appear.
1
SPEAKER_02
Then from perspective of intelligence and consciousness, my current intuition is that they might be quite intertwined.
1
SPEAKER_02
First of all, it might not be that it s like a binary thing that you have intelligence or consciousness.
1
SPEAKER_02
It seems to be more like a continuous component.
1
SPEAKER_02
Let s see, if we look, for instance, on the event networks recognizing images, people are able to show that the activations of these networks correlate very strongly with activations in visual cortex of some monkeys.
1
SPEAKER_02
The same seems to be true about language models.
1
SPEAKER_02
Also, if you, for instance, look, if you train agent in 3D world, at first, you know, it barely recognizes what is going on.
1
SPEAKER_02
Over the time, it kind of recognizes foreground from background.
1
SPEAKER_02
Over the time, it kind of knows where there is a foot.
1
SPEAKER_02
and it just follows it.
1
SPEAKER_02
Over the time, it actually starts having a 3D perception.
1
SPEAKER_02
So it is possible, for instance, to look inside of the head of an agent and ask, what would it see if it looks to the right?
1
SPEAKER_02
And the crazy thing is, you know, initially, when the agents are barely trained, these predictions are pretty bad.
1
SPEAKER_02
Over the time, they become better and better.
1
SPEAKER_02
You can still see
1
SPEAKER_02
that if you ask what happens when the head is turned by 360 degrees, for some time they think that the different thing appears and then at some stage they understand actually that the same thing supposed to appear.
1
SPEAKER_02
So they get like a understanding of 3D structure.
1
SPEAKER_02
It s also, you know, very likely that they have inside some
1
SPEAKER_02
level of symbolic reasoning, particularly symbols for other agents.
1
SPEAKER_02
So when you look at DOTA agents, they collaborate together and they have some anticipation of
1
SPEAKER_02
if they would win battle, they have some expectations with respect to other agents.
1
SPEAKER_02
I might be, you know, too much anthropomorphizing how the things look for me.
1
SPEAKER_02
But the fact that they have a symbol for other agents,
1
SPEAKER_02
makes me believe that at some stage as they are optimizing for skills, they would have also symbol to describe themselves.
1
SPEAKER_02
This is like a very useful symbol to have.
1
SPEAKER_02
And this particularly, I would call it like a self-consciousness or self-awareness.
1
SPEAKER_02
And still it might be different from the consciousness.
1
SPEAKER_02
So I guess the way how I m understanding the word consciousness,
1
SPEAKER_02
Let s say the experience of drinking a coffee or let s say experience of being a bat.
1
SPEAKER_02
That s the meaning of the word consciousness.
1
SPEAKER_02
It doesn t mean to be awake.
1
SPEAKER_02
Yeah, it feels it might be also somewhat related to memory and recurrent connections.
1
SPEAKER_02
So it s kind of like if you look at anesthetic drugs, they might be like they essentially they disturb brain waves such that
1
SPEAKER_02
Maybe memory is not not formed.
1
SPEAKER_00
So there s a lessening of consciousness when you do that.
1
SPEAKER_00
Correct.
1
SPEAKER_00
And so that s one way to intuit what is consciousness.
1
SPEAKER_00
There s also kind of another.
1
SPEAKER_00
element here, it could be that it s, you know, this kind of self-awareness module that you described, plus the actual subjective experience is a storytelling module that tells us a story about
1
SPEAKER_00
what we re experiencing.
1
SPEAKER_02
The crazy thing.
1
SPEAKER_02
So let s say, I mean, in meditation, they teach people not to speak story inside of the head.
1
SPEAKER_02
And there is also some fraction of population who doesn t have actually narrator.
1
SPEAKER_02
I know people who don t have a narrator.
1
SPEAKER_02
And they have to use external people in order to kind of
1
SPEAKER_02
solve tasks that require internal narrator.
1
SPEAKER_02
So it seems that it s possible to have the experience without the talk.
1
SPEAKER_00
What are we talking about when we talk about the internal narrator?
1
SPEAKER_02
Is that the voice when you re like, yeah, I thought that that s what you are referring to.
1
SPEAKER_00
Well, I was referring more on it like not an actual voice.
1
SPEAKER_00
I meant like there s some kind of like subjective experience feels like it s fundamentally about storytelling to ourselves.
1
SPEAKER_00
It feels like the feeling is a story that is much
1
SPEAKER_00
much simpler abstraction than the raw sensory information.
1
SPEAKER_00
So there feels like it s a very high level of abstraction that is useful for me to feel like entity in this world.
1
SPEAKER_00
Most useful aspect of it is that because I m conscious, I think there s an intricate connection to me not wanting to die.
1
SPEAKER_00
So like, it s a useful hack to really prioritize not dying.
1
SPEAKER_00
Like those seem to be somewhat connected.
1
SPEAKER_00
So I m telling this story of like, it s richly feels like something to be me.
1
SPEAKER_00
And the fact that me exists in this world, I want to preserve me.
1
SPEAKER_00
And so that makes it a useful agent hack.
1
SPEAKER_02
So I will just refer maybe to the first part, as you said, about that kind of story of describing who you are.
1
SPEAKER_02
I was thinking about it even so, you know, obviously, I like thinking about consciousness.
1
SPEAKER_02
I like thinking about AI as well.
1
SPEAKER_02
And I m trying to see analogies of these things in AI.
1
SPEAKER_02
What would it correspond to?
1
SPEAKER_02
So.
1
SPEAKER_02
OpenAI trained a model called GPT, which can generate pretty amusing text on arbitrary topic.
1
SPEAKER_02
And one way to control GPT is by putting into prefix at the beginning of the text some information, what would be the story about.
1
SPEAKER_02
You can have even chat with, you know, with GPT by saying that the chat is with Lex or Elon Musk or so.
1
SPEAKER_02
And GPT would just pretend to be you or Elon Musk or so.
1
SPEAKER_02
And it almost feels that this story that we give ourselves to describe our life, it s almost like
1
SPEAKER_02
things that you put into context of GPT.
1
SPEAKER_02
Yeah.
1
SPEAKER_02
And it generates the... But the context we provide to GPT is multimodal.
1
SPEAKER_02
So GPT itself is multimodal.
1
SPEAKER_02
GPT itself hasn t learned actually from experience of single human, but from the experience of humanity.
1
SPEAKER_02
It s a chameleon.
1
SPEAKER_02
You can turn it into anything.
1
SPEAKER_02
And in some sense, by providing context, it behaves as the thing that you wanted it to be,
1
SPEAKER_02
And it s interesting that people have stories of who they are.
1
SPEAKER_02
And as you said, these stories, they help them to operate in the world.
1
SPEAKER_02
But it s also interesting, I guess, various people find it out through meditation or so, that there might be some patterns that you have learned when you were a kid that actually are not serving you anymore.
1
SPEAKER_02
And you also might be thinking that that s who you are, and that s actually just a story.
1
SPEAKER_00
Yes, it s a useful hack, but sometimes it gets us into trouble.
1
SPEAKER_00
It s a local Optima.
1
SPEAKER_02
It s a local Optima.
1
SPEAKER_00
You wrote that Stephen Hawking, he tweeted, Stephen Hawking asked, what breathes fire into equations?
1
SPEAKER_00
Which meant, what makes given mathematical equations realize the physics of a universe?
1
SPEAKER_00
Similarly, I wonder what breathes fire into computation?
1
SPEAKER_00
What makes given computation conscious?
1
SPEAKER_00
Okay, so how do we engineer consciousness?
1
SPEAKER_00
How do you breathe fire and magic into the machine?
1
SPEAKER_02
So it seems clear to me that not every computation is conscious.
1
SPEAKER_02
I mean, you can, let s say, just keep on multiplying one matrix over and over again.
1
SPEAKER_02
And it might be a gigantic matrix.
1
SPEAKER_02
You can put a lot of computation.
1
SPEAKER_02
I don t think it would be conscious.
1
SPEAKER_02
So in some sense, the question is, what are the computations which could be conscious?
1
SPEAKER_02
I mean, so one assumption is that it has to do purely with computation, that you can abstract away matter.
1
SPEAKER_02
Other possibilities that it s very important was the realization of computation that it has to do with some force fields or so, and they bring consciousness.
1
SPEAKER_02
At the moment, my intuition is that it can be fully abstracted away.
1
SPEAKER_02
So in case of computation, you can ask yourself, what are the mathematical objects or so that could bring such properties?
1
SPEAKER_02
So for instance,
1
SPEAKER_02
If we think about the models, AI models, what they truly try to do, or models like GPT, is they try to predict next word or so.
1
SPEAKER_02
And this turns out to be equivalent to compressing text.
1
SPEAKER_02
And because in some sense, compression means that you learn the model of reality and you have just to
1
SPEAKER_02
remember where are your mistakes, the better you are in predicting.
1
SPEAKER_02
And in some sense, when we look at our experience, also when you look, for instance, at the car driving, you know in which direction it will go.
1
SPEAKER_02
You are good like in prediction.
1
SPEAKER_02
You know, it might be the case that the consciousness is intertwined with compression.
1
SPEAKER_02
It might be also the case that self-consciousness has to do with compressor trying to compress itself.
1
SPEAKER_02
So I was just wondering, what are the objects in mathematics or computer science which are mysterious that could have to do with consciousness?
1
SPEAKER_02
And then I thought,
1
SPEAKER_02
You know, you see in mathematics, there is something called Gadiel theorem, which means if you have sufficiently complicated mathematical system, it is possible to point the mathematical system back on itself.
1
SPEAKER_02
In computer science, there is something called helping problem.
1
SPEAKER_02
It s somewhat similar construction.
1
SPEAKER_02
So I thought that, you know, if we believe that under assumption that consciousness has to do with compression,
1
SPEAKER_02
then you could imagine that as you keep on compressing things, then at some point it actually makes sense for the compressor to compress itself.
1
SPEAKER_02
Metacompression.
1
SPEAKER_02
Yeah.
1
SPEAKER_00
Consciousness is metacompression.
1
SPEAKER_02
That s an idea.
1
SPEAKER_02
And in some sense, you know, the crazy... I love it.
1
SPEAKER_00
Thank you.
1
SPEAKER_00
But do you think, if we think of a Turing machine, a universal Turing machine, can that achieve consciousness?
1
SPEAKER_00
So is there something beyond our traditional definition of computation that s required?
1
SPEAKER_02
So it s a specific computation, and I said this computation has to do with compression.
1
SPEAKER_02
And the compression itself, maybe other way of putting it is like you are internally creating the model of reality.
1
SPEAKER_02
In order, like you try inside to simplify reality in order to predict what s going to happen.
1
SPEAKER_02
And that also feels somewhat similar to how I think actually about my own conscious experience.
1
SPEAKER_02
So clearly, I don t have access to reality.
1
SPEAKER_02
The only access to reality is through cable going to my brain.
1
SPEAKER_02
And my brain is creating a simulation of reality.
1
SPEAKER_02
And I have access to the simulation of reality.
1
SPEAKER_00
Are you by any chance aware of the Hutter Prize, Marcus Hutter?
1
SPEAKER_00
He made this prize for compression of Wikipedia pages.
1
SPEAKER_00
And there s a few qualities to it.
1
SPEAKER_00
One, I think, has to be perfect compression, which makes
1
SPEAKER_00
I think that little quirk makes it much less applicable to the general task of intelligence, because it feels like intelligence is always going to be messy.
1
SPEAKER_00
Like perfect compression feels like it s not the right goal, but it s nevertheless a very interesting goal.
1
SPEAKER_00
So for him, intelligence equals compression.
1
SPEAKER_00
And so the smaller you make the file, given a large
1
SPEAKER_00
Wikipedia page, the more intelligent the system has to be.
1
SPEAKER_02
Yeah, that makes sense.
1
SPEAKER_02
So you can make perfect compression if you store errors.
1
SPEAKER_02
And I think that actually what he meant is you have algorithm plus errors.
1
SPEAKER_02
By the way, Hutter is a, he was PhD advisor of Schoenleck, who is a DeepMind co-founder.
1
SPEAKER_00
Yeah, yeah.
1
SPEAKER_00
So there s an interesting, and now he s a DeepMind, there s an interesting network of people.
1
SPEAKER_00
He s one of the people that,
1
SPEAKER_00
I think, seriously took on the task of what would an AGI system look like?
1
SPEAKER_00
I think for a longest time, the question of AGI was not taken seriously or rather rigorously.
1
SPEAKER_00
And he did just that.
1
SPEAKER_00
Like, mathematically speaking, what would the model look like?
1
SPEAKER_00
If you remove the constraints of it having to be
1
SPEAKER_00
having to have a reasonable amount of memory, reasonable amount of running time complexity, computation time, what would it look like?
1
SPEAKER_00
And essentially it s a half math, half philosophical discussion of how would like a reinforcement learning type of framework look like for an AGI.
1
SPEAKER_02
Yeah, so he developed a framework even to describe what s optimal with respect to reinforcement learning.
1
SPEAKER_02
Like there is a theoretical framework, which is, as you said, under assumption, there is infinite amount of memory and compute.
1
SPEAKER_02
And there was actually one person before his name is Solomonov, who then extended Solomonov work to reinforcement learning.
1
SPEAKER_02
But there exists a theoretical algorithm, which is optimal algorithm to build intelligence.
1
SPEAKER_02
And I can actually explain the algorithm.
1
SPEAKER_02
Yes.
1
SPEAKER_00
Let s go.
1
SPEAKER_00
Let s go.
1
SPEAKER_02
So the task itself.
1
SPEAKER_00
Can I just pause how absurd it is for brain in a skull trying to explain the algorithm for intelligence?
1
SPEAKER_00
Just go ahead.
1
SPEAKER_02
It is pretty crazy.
1
SPEAKER_02
It is pretty crazy that, you know, the brain itself is actually so small and it can ponder how to design algorithms that optimally solve the problem of intelligence.
1
SPEAKER_00
OK.
1
SPEAKER_00
So what s the algorithm?
1
SPEAKER_02
So let s see.
1
SPEAKER_02
So first of all, the task itself is described as you have infinite sequence of zeros and ones.
1
SPEAKER_02
Okay, you read n bits, and you re about to predict n plus one bit.
1
SPEAKER_02
So that s the task.
1
SPEAKER_02
And you could imagine that every task could be casted as such a task.
1
SPEAKER_02
So if, for instance, you have images and labels, you can just turn every image into a sequence of zeros and ones, then label, you concatenate labels, and that s actually, you could start by having training data first, and then afterwards you have test data.
1
SPEAKER_02
So theoretically, any problem could be casted as a problem of predicting zeros and ones on this infinite tape.
1
SPEAKER_02
So let s say you read already n bits, and you want to predict n plus 1 bit.
1
SPEAKER_02
And I will ask you to write every possible program that generates these n bits.
1
SPEAKER_02
Okay, so, and you can have, you choose programming language, it can be Python or C++, and the difference between programming languages might be, there is a difference by constant, asymptotically, your predictions will be equivalent.
1
SPEAKER_02
So you read n bits, you enumerate all the programs that produce these n bits in their output.
1
SPEAKER_02
And then in order to predict n plus 1 bit, you actually weight the programs according to their length.
1
SPEAKER_02
And there is some specific formula how you weight them.
1
SPEAKER_02
And then the n plus 1 bit prediction is the prediction from each of these programs according to that weight.
1
SPEAKER_00
Like statistically.
1
SPEAKER_00
Statistically, yeah.
1
SPEAKER_00
So the smaller the program, the more likely you are to pick its output.
1
SPEAKER_00
So that algorithm is grounded in the hope or the intuition that the simple answer is the right one.
1
SPEAKER_02
It s a formalization of it.
1
SPEAKER_02
It also means like if you would ask the question, after how many years would, you know, Sun explode?
1
SPEAKER_02
You can say, it s more likely the answer is due to some power, because it s a shorter program.
1
SPEAKER_00
Yeah.
1
SPEAKER_00
Well, I don t have a good intuition about how different the space of short programs are from the space of large programs.
1
SPEAKER_00
Like, what is the universe where short programs run things?
1
SPEAKER_02
So as I said, the things have to agree with nbits.
1
SPEAKER_02
So even if you have, you need to start, okay, if you have very short program and they re like, still some has, if it s not perfectly prediction of nbits, you have to start errors.
1
SPEAKER_02
What are the errors?
1
SPEAKER_02
And that gives you the full program that agrees on nbits.
1
SPEAKER_02
Oh, so you don t agree perfectly with the end bits and you store.
1
SPEAKER_02
That s like a longer, longer program, slightly longer program, because it contains these extra bits of errors.
1
SPEAKER_02
That s fascinating.
1
SPEAKER_00
What s what s your intuition about the programs that are able to do cool stuff like intelligence and consciousness?
1
SPEAKER_00
Are they perfectly like is it is there if then statements in them?
2
SPEAKER_01
So is there a lot of exceptions that they re storing?
2
SPEAKER_01
So you could imagine if there would be tremendous amount of if statements, then they wouldn t be that short.
2
SPEAKER_01
In case of neural networks, you could imagine that what happens is when you start with uninitialized neural network,
2
SPEAKER_01
It stores internally many possibilities how the problem can be solved.
2
SPEAKER_01
And SGD is kind of magnifying some paths which are slightly similar to the correct answer.
2
SPEAKER_01
So it s kind of magnifying correct programs.
2
SPEAKER_01
And in some sense, HDD is a search algorithm in the program space.
2
SPEAKER_01
And the program space is represented by, you know, kind of the wiring inside of the neural network.
2
SPEAKER_01
And there s like an insane number of ways how the features can be computed.
2
SPEAKER_00
Let me ask you the high level basic question that s not so basic.
2
SPEAKER_00
What is deep learning?
2
SPEAKER_00
Is there a way you d like to think of it that is different than like a generic textbook definition?
2
SPEAKER_01
The thing that I hinted just a second ago is maybe the closest to how I m thinking these days about deep learning.
2
SPEAKER_01
So the statement is neural networks can represent some programs.
2
SPEAKER_01
It seems that various modules that we are actually adding up to, or like we want networks to be deep because we want multiple steps of the computation.
2
SPEAKER_01
And deep learning provides the way to represent space of programs, which is searchable.
2
SPEAKER_01
And it s searchable with stochastic gradient descent.
2
SPEAKER_01
So we have an algorithm to search over a humongous number of programs.
2
SPEAKER_01
And gradient descent kind of bubbles up the things that tend to give correct answers.
2
SPEAKER_00
So a neural network with fixed weights that s optimized, do you think of that as a single program?
2
SPEAKER_01
So there is a work by Christopher Olach where he, so he works on interpretability of neural networks.
2
SPEAKER_01
And he was able to identify inside of the neural network, for instance, a detector of a wheel for a car or the detector of a mask for a car.
2
SPEAKER_01
And then he was able to separate them out and assemble them together using a simple program for the detector, for a car detector.
2
SPEAKER_00
That s like, if you think of traditionally defined programs, that s like a function within a program that this particular neural network was able to find.
2
SPEAKER_00
And you can tear that out just like you can copy and paste from Stack Overflow.
2
SPEAKER_00
So any program is a composition of smaller programs.
2
SPEAKER_01
Yeah, I mean, the nice thing about the neural networks is that it allows the things to be more fuzzy than in case of programs.
2
SPEAKER_01
In case of programs, you have this like a branching this way or that way.
2
SPEAKER_01
And the neural networks, they have an easier way to be somewhere in between or to share things.
2
SPEAKER_00
What to use the most beautiful or surprising idea in deep learning and the utilization of these neural networks, which by the way, for people who are not familiar, neural networks is a bunch of, what would you say?
2
SPEAKER_00
It s inspired by the human brain.
2
SPEAKER_00
There s neurons, there s connection between those neurons, there s inputs and there s outputs and there s millions or billions of those neurons.
2
SPEAKER_00
And the learning happens
2
SPEAKER_00
by adjusting the weights on the edges that connect these neurons.
2
SPEAKER_01
Thank you for giving definition.
2
SPEAKER_01
I supposed to do it, but I guess you have enough empathy to listeners to actually know that that might be useful.
2
SPEAKER_00
No, that s like...
2
SPEAKER_00
So I m asking Plato of like, what is the meaning of life?
2
SPEAKER_00
He s not going to answer.
2
SPEAKER_00
You re being philosophical and deep and quite profound talking about the space of programs, which is just very interesting.
2
SPEAKER_00
But also for people who are just not familiar with the hell we re talking about when we talk about deep learning.
2
SPEAKER_00
Anyway, sorry.
2
SPEAKER_00
What is the most beautiful or surprising idea to you in all the time you ve worked in deep learning?
2
SPEAKER_00
And you worked on a lot of
2
SPEAKER_00
fascinating projects, applications of neural networks.
2
SPEAKER_00
It doesn t have to be big and profound.
2
SPEAKER_00
It can be a cool trick.
2
SPEAKER_01
Yeah, I mean, I m thinking about the trick, but it s still amusing to me that it works at all.
2
SPEAKER_01
Yeah.
2
SPEAKER_01
That let s say that the extremely simple algorithm, stochastic gradient descent, which is something that I would be able to derive on the piece of paper to high school student, when put at the scale of thousands of machines, actually can create the behaviors which we called kind of human-like behaviors.
2
SPEAKER_00
So in general, any application of stochastic gradient descent in neural networks is amazing to you.
2
SPEAKER_00
Or is there a particular application in natural language, reinforcement learning?
2
SPEAKER_00
And also, what do you attribute?
2
SPEAKER_00
that success to, is it just scale?
2
SPEAKER_00
What profound insight can we take from the fact that the thing works for gigantic sets of variables?
2
SPEAKER_01
I mean, the interesting thing is these algorithms, they were invented decades ago, and people actually gave up on the idea.
2
SPEAKER_01
Yeah.
2
SPEAKER_01
You know, back then they thought that we need profoundly different algorithms and they spent a lot of cycles on very different algorithms.
2
SPEAKER_01
And I believe that, you know, we have seen that various innovations, let s say like Transformer or Dropout or so they can, you know, vastly help.
2
SPEAKER_01
But it s also remarkable to me that this algorithm from 60s or so, or I mean, you can even say that the gradient descent was invented by Leibniz in, I guess, 18th century or so.
2
SPEAKER_01
That actually is the core of learning.
2
SPEAKER_01
In the past, people are, it s almost like a, out of the maybe an ego, people are saying that it cannot be the case that such a simple algorithm is the, you know, could solve complicated problems.
2
SPEAKER_01
So they were in search for the other algorithms.
2
SPEAKER_01
And as I m saying, like, I believe that actually we are in the game where there is, there are actually, frankly, three levers.
2
SPEAKER_01
There is compute, there are algorithms, and there is data.
2
SPEAKER_01
And if we want to build intelligent systems, we have to pull all three levers and they are actually multiplicative.
2
SPEAKER_01
And it s also interesting, so you ask, is it only compute?
2
SPEAKER_01
People internally, they did the studies to determine how much gains they were coming from different levers.
2
SPEAKER_01
And so far, we have seen that more gains came from compute than algorithms.
2
SPEAKER_01
But also, we are in the world that in case of compute, there is a kind of exponential increase in funding.
2
SPEAKER_01
And at some point, it s impossible to invest more.
2
SPEAKER_01
It s impossible to invest $10 trillion.
2
SPEAKER_01
as we are speaking about, let s say, all taxes in US.
2
SPEAKER_00
But you re talking about money that could be innovation in the compute.
2
SPEAKER_01
That s that s true as well.
2
SPEAKER_01
So, I mean, they re like a few pieces.
2
SPEAKER_01
So one piece is human brain is an incredible supercomputer.
2
SPEAKER_01
And they re like a. It has 100 trillion parameters.
2
SPEAKER_01
Or if you try to count various quantities in the brain, there are neurons, synapses.
2
SPEAKER_01
There are a small number of neurons.
2
SPEAKER_01
There is a lot of synapses.
2
SPEAKER_01
It s unclear even how to map synapses to parameters of neural networks.
2
SPEAKER_01
But it s clear that there are many more.
2
SPEAKER_01
So it might be the case that our networks are still somewhat small.
2
SPEAKER_01
It also might be the case that they are more efficient than brain or less efficient by some huge factor.
2
SPEAKER_01
I also believe that there will be, at the moment, we are at the stage that these neural networks, they require 1,000x or a huge factor of more data than humans do.
2
SPEAKER_01
And it will be a matter of, there will be algorithms that
2
SPEAKER_01
a vastly decreased sample complexity, I believe so.
2
SPEAKER_01
But the place where we are heading today is there are domains which contains million X more data.
2
SPEAKER_01
And even though computers might be 1,000 times slower than humans in learning, that s not a problem.
2
SPEAKER_01
For instance, I believe that it should be possible to create superhuman therapies
2
SPEAKER_01
And there are even simple steps of doing it.
2
SPEAKER_01
And the core reason is there is just machine will be able to read way more transcripts of therapies.
2
SPEAKER_01
And then it should be able to speak simultaneously with many more people.
2
SPEAKER_01
And it should be possible to optimize it all in parallel.
2
SPEAKER_00
And well, there s now you re touching on something I deeply care about and think is way harder than we imagine.
2
SPEAKER_00
What s the goal of a therapist?
2
SPEAKER_00
What s the goal of therapist?
2
SPEAKER_01
Hmm.
2
SPEAKER_00
So, okay, so one goal, now this is terrifying to me, but there s a lot of people that contemplate suicide, self-affirmed depression, and they could significantly be helped with therapy.
2
SPEAKER_00
And the idea that an AI algorithm might be in charge of that, it s like a life and death task.
2
SPEAKER_00
It s a,
2
SPEAKER_00
The stakes are high.
2
SPEAKER_00
So one goal for a therapist, whether human or AI, is to prevent suicide ideation, to prevent suicide.
2
SPEAKER_00
How do you achieve that?
2
SPEAKER_01
So let s see.
2
SPEAKER_01
So to be clear, I don t think that the current models are good enough for such a task because it requires insane amount of understanding, empathy, and the models are far from this place, but it s... But do you think that understanding empathy, that signal is in the data?
2
SPEAKER_01
I think there is some signal in the data, yes.
2
SPEAKER_01
I mean, there are plenty of transcripts of conversations and it is possible to, it is possible from it to understand personalities.
2
SPEAKER_01
It is possible from it to understand if conversation is friendly, amicable, antagonistic.
2
SPEAKER_01
It is, I believe that the, you know, given the fact that the models that we train now, they can have,
2
SPEAKER_01
they are chameleons that they can have any personality, they might turn out to be better in understanding the personality of other people than anyone else.
2
SPEAKER_01
And they should be empathetic.
2
SPEAKER_01
Yeah.
2
SPEAKER_00
Interesting.
2
SPEAKER_00
But
2
SPEAKER_00
I wonder if there s some level of multiple modalities required to be able to be empathetic as a human experience, whether language is not enough to understand death, to understand fear, to understand childhood trauma.
2
SPEAKER_00
to understand wit and humor required when you re dancing with a person who might be depressed or suffering, both humor and hope and love and all those kinds of things.
2
SPEAKER_00
So there s another underlying question, which is self-supervised versus supervised.
2
SPEAKER_00
So can you get that?
2
SPEAKER_00
from the data by just reading a huge number of transcripts?
2
SPEAKER_01
So I think that reading a huge number of transcripts is a step one.
2
SPEAKER_01
It s like the same way as you cannot learn to dance just from YouTube.
2
SPEAKER_01
By watching it, you have to actually try it out yourself.
2
SPEAKER_01
So I think that here, that s a similar situation.
2
SPEAKER_01
I also wouldn t deploy the system in the high-stakes situations right away.
2
SPEAKER_01
but kind of see gradually where it goes.
2
SPEAKER_01
And obviously, initially, it would have to go hand in hand with humans.
2
SPEAKER_01
But at the moment, we are in the situation that actually there is many more people who actually would like to have a therapy or speak with someone than there are therapies out there.
2
SPEAKER_01
So fundamentally, I was thinking, what are the things that can vastly increase people well-being?
2
SPEAKER_01
Therapy is one of them.
2
SPEAKER_01
I think meditation is other one.
2
SPEAKER_01
I guess maybe human connection is a third one.
2
SPEAKER_01
And I guess
2
SPEAKER_01
Pharmacologically, it s also possible.
2
SPEAKER_01
Maybe direct brain stimulation or something like that, but these are pretty much options out there.
2
SPEAKER_01
Then let s say the way I m thinking about the AGI endeavor is by default, that s an endeavor to increase amount of wealth.
2
SPEAKER_01
And I believe that we can vastly increase amount of wealth for everyone.
2
SPEAKER_01
And simultaneously, so I mean, these are like two endeavors that make sense to me.
2
SPEAKER_01
One is like essentially increase amount of wealth.
2
SPEAKER_01
And second one is increase overall human well-being.
2
SPEAKER_01
And those are coupled together.
2
SPEAKER_01
And they can, like I would say, these are different topics.
2
SPEAKER_01
One can help another.
2
SPEAKER_00
And, you know, therapist is a funny word because I see friendship and love as therapy.
2
SPEAKER_00
I mean, so therapist broadly defined is just friendship, is a friend.
2
SPEAKER_00
So like therapist has a very kind of clinical sense to it.
2
SPEAKER_00
But what is human connection?
2
SPEAKER_00
You re like, not to get all Camus and Dostoevsky on you, but you know, life is suffering and we draw, we seek connection with other humans as we desperately try to make sense of this world in the deep, overwhelming loneliness that we feel inside.
2
SPEAKER_01
So I think connection has to do with understanding.
2
SPEAKER_01
And I think that almost like a lack of understanding causes suffering.
2
SPEAKER_01
If you speak with someone and you do you feel ignored, that actually causes pain.
2
SPEAKER_01
If you are feeling deeply understood, that actually they might not even tell you what to do in life, but like a pure understanding or just being heard.
2
SPEAKER_00
Understanding is a kind of it s a lot, you know, just being heard.
2
SPEAKER_00
Feel like you re being heard.
2
SPEAKER_00
Like somehow that s alleviation temporarily of the loneliness.
2
SPEAKER_00
That if somebody knows you re here with their body language, with the way they are, with the way they look at you, with the way they talk, you feel less alone for a brief moment.
2
SPEAKER_01
Yeah, very much agree.
2
SPEAKER_01
So I thought in the past about a somewhat similar question to yours, which is what is love?
2
SPEAKER_01
Rather, what is connection?
2
SPEAKER_01
Yes.
2
SPEAKER_01
And obviously, I think about these things from an AI perspective, what would it mean?
2
SPEAKER_01
So I said that the
2
SPEAKER_01
You know, intelligence has to do with some compression, which is more or less, like I can say, almost understanding of what is going around.
2
SPEAKER_01
It seems to me that other aspect is there seem to be reward functions.
2
SPEAKER_01
And you can have a reward for food, for maybe human connection, for let s say warmth, sex, and so on.
2
SPEAKER_01
And it turns out that various people might be optimizing slightly different reward functions.
2
SPEAKER_01
They essentially might care about different things.
2
SPEAKER_01
And in case of love, at least the love between two people, you can say that the boundary between people dissolves to such extent that they end up optimizing each other reward functions.
2
SPEAKER_00
Yeah.
2
SPEAKER_00
Oh, that s interesting.
2
SPEAKER_00
Celebrate the success of each other.
2
SPEAKER_01
Yeah.
2
SPEAKER_01
In some sense, I would say love means helping others to optimize their reward functions, not your reward functions, not the things that you think are important, but the things that the person cares about.
2
SPEAKER_01
you try to help them to optimize it.
2
SPEAKER_00
So love is, if you think of two reward functions, you just, it s a condition.
2
SPEAKER_00
Yeah.
2
SPEAKER_00
You combine them together.
2
SPEAKER_01
Yeah, pretty much.
2
SPEAKER_00
Maybe like with a weight and it depends like the dynamic of the relationship.
2
SPEAKER_01
Yeah, I mean, you could imagine that if you are fully optimizing someone s reward function without yours, then maybe you are creating codependency or something like that.
2
SPEAKER_01
I m not sure what s the appropriate way.
2
SPEAKER_01
But the interesting thing is, I even think that the individual person, we ourselves, we are actually less of a unified insight.
2
SPEAKER_01
So for instance, if you look at the donut,
2
SPEAKER_01
On one level, you might think, oh, this looks tasty.
2
SPEAKER_01
I would like to eat it.
2
SPEAKER_01
On another level, you might tell yourself, I shouldn t be doing it because I want to gain muscles.
2
SPEAKER_01
And you might do it regardless, kind of against yourself.
2
SPEAKER_01
So it seems that even within ourselves, they re almost like a kind of intertwined personas.
2
SPEAKER_01
And I believe that the self-love
2
SPEAKER_01
means that the love between all these persons, which also means being able to love yourself when we are angry or stressed or so.
2
SPEAKER_00
Combining all those reward functions of the different selves you have.
2
SPEAKER_01
Yeah.
2
SPEAKER_01
And accepting that they are there.
2
SPEAKER_01
Like, you know, often people, they have a negative self talk or they say, I don t like when I m angry.
2
SPEAKER_01
And like I try to imagine
2
SPEAKER_01
Try to imagine if there would be like a small baby Lex, like five years old, angry.
2
SPEAKER_01
And then you re like, you shouldn t be angry, like stop being angry.
2
SPEAKER_01
But like instead actually you want Lex to come over, give him a hug.
2
SPEAKER_01
And it s like I say, it s fine.
2
SPEAKER_01
You can be angry as long as you want.
2
SPEAKER_01
And then he would stop.
2
SPEAKER_00
Or maybe not.
2
SPEAKER_01
Or maybe not, but you cannot expect it even.
2
SPEAKER_00
Yeah.
2
SPEAKER_00
But still, that doesn t explain the why of love.
2
SPEAKER_00
Like, why is love part of the human condition?
2
SPEAKER_00
Why is it useful to combine the reward functions?
2
SPEAKER_00
It seems like that doesn t, I mean, I don t think reinforcement learning frameworks can give us
3
SPEAKER_01
answers to why even the Hutter framework has an objective function that s static.
3
SPEAKER_01
So we came to existence as a consequence of evolutionary process.
3
SPEAKER_01
And in some sense, the purpose of evolution is survival.
3
SPEAKER_01
And then this complicated optimization objective baked into us, let s say, compression, which might help us operate in the real world.
3
SPEAKER_01
And it baked into us various reward functions.
3
SPEAKER_00
Yeah.
3
SPEAKER_01
And then to be clear, at the moment we are operating in the regime which is somewhat out of distribution where the event evolution optimizes.
3
SPEAKER_01
It s almost like love is a consequence of cooperation that we ve discovered is useful.
3
SPEAKER_01
Correct.
3
SPEAKER_01
In some way, it s even the case if you... I just love the idea that love is like the out of distribution.
3
SPEAKER_01
Or it s not out of distribution.
3
SPEAKER_01
It s like, as you said, it evolved for cooperation.
3
SPEAKER_00
Yes.
3
SPEAKER_01
And I believe that, in some sense, cooperation ends up helping each of us individually.
3
SPEAKER_01
So it makes sense, evolutionary.
3
SPEAKER_01
And there is, in some sense, and you know, love means there is this dissolution of boundaries, that you have a shared reward function.
3
SPEAKER_01
And we evolve to actually identify ourselves with larger groups.
3
SPEAKER_01
So we can identify ourselves with a family.
3
SPEAKER_01
We can identify ourselves with a country, to such extent that people are willing to give away their life for country.
3
SPEAKER_01
And so there is, we are wired actually even for love.
3
SPEAKER_01
And at the moment, I guess, maybe it would be somewhat more beneficial if we would identify ourselves with all the humanity as a whole.
3
SPEAKER_01
So you can clearly see when people travel around the world, when they run into person from the same country, they say, oh, which city are and all this, like all the sudden they,
3
SPEAKER_01
find all these similarities, they find some, they befriended those folks earlier than others.
3
SPEAKER_01
So there is like a sense, some sense of the belonging.
3
SPEAKER_01
And I would say, I think it would be overall a good thing to the world for people to move towards, I think it s even called open individualism, move toward the mindset of a larger and larger groups.
3
SPEAKER_00
The challenge there, that s a beautiful vision and I share it, to expand that circle of empathy, that circle of love towards the entirety of humanity, but then you start to ask, well, where do you draw the line?
3
SPEAKER_00
Because why not expand it to other conscious beings?
3
SPEAKER_00
And then finally, for our discussion, something I think about is why not expand it to AI systems?
3
SPEAKER_00
Like we start respecting each other when the entity on the other side has the capacity to suffer, because then we develop a capacity to sort of empathize.
3
SPEAKER_00
And so I could see AI systems that are interacting with humans.
3
SPEAKER_00
more and more having conscious like displays.
3
SPEAKER_00
So like they display consciousness through language and through other means.
3
SPEAKER_00
And so then the question is like, well, is that consciousness?
3
SPEAKER_00
Because they are acting conscious.
3
SPEAKER_00
And so, you know, the reason we don t like torturing animals.
3
SPEAKER_00
Is because they look like they re suffering when they re tortured.
3
SPEAKER_00
And if I looks like it s suffering,
3
SPEAKER_00
When it s tortured, how is that not?
3
SPEAKER_00
a requiring of the same kind of empathy from us and respect and rights that animals do and other humans do.
3
SPEAKER_01
I think it requires empathy as well.
3
SPEAKER_01
I mean, I would like, I guess, us or humanity also make a progress in understanding what consciousness is, because I don t want just to be speaking about the philosophy, but rather actually make a scientific, to have a, like, you know, there was a time that people thought that there is a force of life
3
SPEAKER_01
and the things that have this force, they are alive.
3
SPEAKER_01
And I think that there is actually a path to understand exactly what consciousness is.
3
SPEAKER_01
And in some sense, it might require essentially putting probes inside of a human brain, what Neuralink does.
3
SPEAKER_00
So the goal there, I mean, there s several things with consciousness that make it a real discipline, which is one is rigorous measurement of consciousness.
3
SPEAKER_00
And then the other is the engineering of consciousness, which may or may not be related.
3
SPEAKER_00
I mean, you could also run into trouble, like for example, in the United States, the Department of DOT, Department of Transportation, and a lot of different places put a value on human life.
3
SPEAKER_00
I think DOT s value is $9 million per person.
3
SPEAKER_00
sort of in that same way, you can get into trouble if you put a number on how conscious a being is.
3
SPEAKER_00
Because then you can start making policy.
3
SPEAKER_00
If a cow is 0.1 or like 10% as conscious as a human, then you can start making calculations and it might get you into trouble.
3
SPEAKER_00
But then again, that might be a very good way to do it.
3
SPEAKER_01
I would like to move to that place that actually we have scientific understanding what consciousness is.
3
SPEAKER_01
And then we ll be able to actually assign value.
3
SPEAKER_01
And I believe that there is even the path for the experimentation in it.
3
SPEAKER_01
We said that, you know, you could put the probes inside of the brain.
3
SPEAKER_01
There is actually a few other things that you could do with devices like Neuralink.
3
SPEAKER_01
So you could imagine that the way even to measure if AI system is conscious is by literally just plugging into the brain.
3
SPEAKER_01
I mean, that assumes that it s kind of easy, but they re plugging into the brain and asking person if they feel that their consciousness expanded.
3
SPEAKER_01
And this direction, of course, has some issues.
3
SPEAKER_01
You can say, you know, if someone takes a psychedelic drug, they might feel that their consciousness expanded, even though that drug itself is not conscious.
3
SPEAKER_01
Right.
3
SPEAKER_00
So like you can t fully trust the self-report of a person saying their their consciousness is expanded or not.
3
SPEAKER_00
Let me ask you a little bit about psychedelics.
3
SPEAKER_00
There s been a lot of excellent research on different psychedelics, psilocybin, MDMA, even DMT.
3
SPEAKER_00
Drugs in general, marijuana too.
3
SPEAKER_00
What do you think psychedelics do to the human mind?
3
SPEAKER_00
It seems they take the human mind to some interesting places.
3
SPEAKER_00
Is that just a little hack, a visual hack, or is there some profound expansion of the mind?
3
SPEAKER_01
So let s see.
3
SPEAKER_01
I don t believe in magic.
3
SPEAKER_01
I believe in science, in causality.
3
SPEAKER_01
Still, let s say, and then as I said, I think that our subjective experience of reality is we live in the simulation run by our brain.
3
SPEAKER_01
And the simulation that our brain runs, they can be very pleasant or very hellish.
3
SPEAKER_01
Drags, they are changing some hyperparameters of the simulation.
3
SPEAKER_01
It is possible, thanks to change of these hyperparameters, to actually look back on your experience and even see that the given things that we took for granted, they are changeable.
3
SPEAKER_01
So they allow to have an amazing perspective.
3
SPEAKER_01
There is also, for instance, the fact that after DMT, people can see the full movie inside of their head.
3
SPEAKER_01
gives me further belief that the brain can generate that full movie, that the brain is actually
3
SPEAKER_01
learning the model of reality to such extent that it tries to predict what s going to happen next.
3
SPEAKER_01
Yeah, very high resolution so it can replay reality.
3
SPEAKER_01
Extremely high resolution.
3
SPEAKER_01
And it s also kind of interesting to me that somehow there seems to be some similarity between these drugs and meditation itself.
3
SPEAKER_01
And I actually started even these days to think about meditation as a psychedelic.
3
SPEAKER_01
Do you practice meditation?
3
SPEAKER_01
I practice meditation.
3
SPEAKER_01
I mean, I went a few times on the retreats and it feels after like after second or third day of meditation.
3
SPEAKER_01
There is almost like a sense of, you know, tripping.
3
SPEAKER_01
What does a meditation retreat entail?
3
SPEAKER_01
So you wake up early in the morning and you meditate for an extended period of time.
3
SPEAKER_01
Alone?
3
SPEAKER_01
Yeah.
3
SPEAKER_01
So it s optimized, even though there are other people, it s optimized for isolation.
3
SPEAKER_01
So you don t speak with anyone.
3
SPEAKER_01
You don t actually look into other people s eyes.
3
SPEAKER_01
And, you know, you sit on that chair.
3
SPEAKER_01
So Vipassana meditation tells you to focus on the breath.
3
SPEAKER_01
So you try to put all attention into breathing in and breathing out.
3
SPEAKER_01
And the crazy thing is that as you focus attention like that, after some time,
3
SPEAKER_01
their stamps start coming back like some memories that you ve completely forgotten.
3
SPEAKER_01
It almost feels like that you have a mailbox and then you know you are just like archiving email one by one.
3
SPEAKER_01
And at some point there is this like amazing feeling of getting to mailbox zero.
3
SPEAKER_01
Zero emails.
3
SPEAKER_01
And it s very pleasant.
3
SPEAKER_01
It s kind of it s
3
SPEAKER_01
crazy to me that once you resolve these inner stories or like inner traumas, then once there is nothing left, the default state of human mind is extremely peaceful and happy.
3
SPEAKER_01
Like some sense, it feels that
3
SPEAKER_01
It feels, at least to me, the way how when I was a child that I can look at any object and it s very beautiful.
3
SPEAKER_01
I have a lot of curiosity about the simple things.
3
SPEAKER_01
And that s where usually meditation takes me.
3
SPEAKER_00
Are you, what are you experiencing?
3
SPEAKER_00
Are you just taking in simple sensory information and they re just enjoying the rawness of that sensory information?
3
SPEAKER_00
So there s no, there s no memories or all that kind of stuff.
3
SPEAKER_00
You re just enjoying being.
3
SPEAKER_01
Yeah, pretty much.
3
SPEAKER_01
I mean, still there is a... thoughts are slowing down.
3
SPEAKER_01
Sometimes they pop up, but it s also somehow the extended meditation takes you to the space that they are way more friendly, way more positive.
3
SPEAKER_01
There is also this thing that we ve... it almost feels that
3
SPEAKER_01
It almost feels that we are constantly getting a little bit of a reward function, and we are just spreading this reward function on various activities.
3
SPEAKER_01
But if you stay still for an extended period of time, it kind of accumulates, accumulates, accumulates.
3
SPEAKER_01
And there is a sense that at some point it passes some threshold, and it feels as
3
SPEAKER_01
drop is falling into kind of ocean of love and bliss and that s like a this is like a very pleasant and that s why I m saying like a
3
SPEAKER_01
That corresponds to the subjective experience.
3
SPEAKER_01
Some people, I guess, in spiritual community, they describe it that that s the reality.
3
SPEAKER_01
And I would say I believe that there are all sorts of subjective experience that one can have.
3
SPEAKER_01
And I believe that, for instance, meditation might take you to the subjective experiences, which are very pleasant, collaborative.
3
SPEAKER_01
And I would like the world to move toward a more collaborative place.
3
SPEAKER_01
Yeah, I would say that s very pleasant and I enjoy doing stuff like that.
3
SPEAKER_00
I wonder how that maps to your mathematical model of love with the reward function combining a bunch of things.
3
SPEAKER_00
it seems like our life then is we re just, we have this reward function and we re accumulating a bunch of stuff in it with weights.
3
SPEAKER_00
It s like multi-objective.
3
SPEAKER_00
And what meditation is, is you just remove them, remove them until the weight on one or just a few is very high.
3
SPEAKER_00
And that s where the pleasure comes from.
3
SPEAKER_01
Yeah, so something similar to how I m thinking about this.
3
SPEAKER_01
So I told you that there is this like, there is a story of who you are.
3
SPEAKER_01
And I think almost about it as a, you know, text prepended to GPT.
3
SPEAKER_01
And some people refer to it as ego.
3
SPEAKER_01
It s like a story who you are.
3
SPEAKER_01
So ego is the prompt for GPT-3.
3
SPEAKER_00
Yes.
3
SPEAKER_01
And that s the description of you.
3
SPEAKER_01
And then with meditation, you can get to the point that actually you experience things without the prompt.
3
SPEAKER_01
And you experience things as they are.
3
SPEAKER_01
You are not biased over the description how they re supposed to be.
3
SPEAKER_01
That s very pleasant.
3
SPEAKER_01
And then with respect to the reward function, it s possible to get to the point that there is this solution of self.
3
SPEAKER_01
And therefore, you can say that you are having your brain attempt to simulate the reward function of everyone else or everything.
3
SPEAKER_01
There is this love which feels like a oneness with everything.
3
SPEAKER_01
And that s also very beautiful, very pleasant.
3
SPEAKER_01
At some point, you might have a lot of altruistic thoughts during that moment, and then the self always comes back.
3
SPEAKER_00
How would you recommend, if somebody is interested in meditation, like a big thing to take on as a project, would you recommend a meditation retreat?
3
SPEAKER_00
How many days?
3
SPEAKER_00
What kind of thing would you recommend?
3
SPEAKER_01
I think that actually retreat is the way to go.
3
SPEAKER_01
And it almost feels that, as I said, like meditation is a psychedelic, but when you take it in the small dose, you might barely feel it.
3
SPEAKER_01
Once you get the high dose, actually you re going to feel it.
3
SPEAKER_00
So even cold turkey, if you haven t really seriously meditated for a long period of time, just go to a retreat.
3
SPEAKER_00
Yeah.
3
SPEAKER_00
How many days?
3
SPEAKER_00
I would start the weekend one.
3
SPEAKER_00
Weekend, so like two, three days.
3
SPEAKER_01
And it s interesting that first or second day it s hard and at some point it becomes easy.
3
SPEAKER_00
There s a lot of seconds in a day.
3
SPEAKER_00
How hard is the meditation retreat just sitting there in a chair?
3
SPEAKER_01
So the thing is actually
3
SPEAKER_01
It literally just depends on your own framing.
3
SPEAKER_01
If you are in the mindset that you are waiting for it to be over, or you are waiting for a nirvana to happen, it will be very unpleasant.
3
SPEAKER_01
And in some sense, even the difficulty, it s not even in the lack of being able to speak with others.
3
SPEAKER_01
You are sitting there, your legs will hurt from sitting.
3
SPEAKER_00
In terms of like the practical things, do you experience kind of discomfort, like physical discomfort of just sitting like your your butt being numb, your legs being sore, all that kind of stuff?
3
SPEAKER_01
Yes, you experience it, and then they teach you to observe it.
3
SPEAKER_01
The crazy thing is, you at first might have a feeling toward trying to escape it.
3
SPEAKER_01
And that becomes very apparent that that s extremely unpleasant.
3
SPEAKER_01
And then you just observe it.
3
SPEAKER_01
And at some point, it just becomes, it just is.
3
SPEAKER_01
It s like, I remember Ilya told me some time ago that he takes a cold shower, and his mindset of taking a cold shower was to embrace suffering.
3
SPEAKER_01
Yeah, excellent.
3
SPEAKER_01
I do the same.
3
SPEAKER_01
That s your style?
3
SPEAKER_01
Yeah, it s my style.
3
SPEAKER_01
I like this.
3
SPEAKER_01
So my style is actually, I also sometimes take cold showers.
3
SPEAKER_01
It is purely observing how the water goes through my body, like purely being present, not trying to escape from there.
3
SPEAKER_01
Yeah.
3
SPEAKER_01
And I would say then it actually becomes pleasant.
3
SPEAKER_00
It s not like, oh, well, that that s interesting.
3
SPEAKER_00
I m also that I mean, that s that s the way to deal with anything really difficult, especially in the physical space is to observe it.
3
SPEAKER_00
To say it s pleasant.
3
SPEAKER_00
Hmm.
3
SPEAKER_00
It s I would use a different word.
3
SPEAKER_00
You re
3
SPEAKER_00
you re accepting of the full beauty of reality, I would say, because stay pleasant.
3
SPEAKER_00
But yeah, in some sense it is pleasant.
3
SPEAKER_00
That s the only way to deal with a cold shower is to become an observer and to find joy in it.
3
SPEAKER_00
Same with like really difficult physical exercise or like running for a really long time, endurance events, just any time you re exhausted, any kind of pain.
3
SPEAKER_00
I think the only way to survive it is not to resist it, it s to observe it.
3
SPEAKER_00
You mentioned Ilya, Ilya Sutskever.
3
SPEAKER_01
He s our chief scientist, but also he s a very close friend of mine.
3
SPEAKER_00
He co-founded OpenAI with you.
3
SPEAKER_00
I ve spoken with him a few times.
3
SPEAKER_00
He s brilliant.
3
SPEAKER_00
I really enjoy talking to him.
3
SPEAKER_00
His mind, just like yours, works in fascinating ways.
3
SPEAKER_00
Both of you are not able to define deep learning simply.
3
SPEAKER_00
What s it like having him as somebody you have technical discussions with in the space of machine learning, deep learning, AI, but also life?
3
SPEAKER_00
What s it like when these two agents get into a self-play situation in a room?
3
SPEAKER_00
What s it like collaborating with him?
3
SPEAKER_01
So I believe that we have extreme respect to each other.
3
SPEAKER_01
So I love Ilya s insight, both like, I guess, about consciousness, life, AI.
3
SPEAKER_00
But in terms of the it s interesting to me because you re a brilliant
4
SPEAKER_00
thinker in the space of machine learning, like intuition, like digging deep in what works, what doesn t, why it works, why it doesn t, and so is Ilya.
4
SPEAKER_00
I m wondering if there s interesting, deep discussions you ve had with him in the past or disagreements that were very productive.
4
SPEAKER_01
So I can say,
4
SPEAKER_01
I also understood over the time where are my strengths.
4
SPEAKER_01
So obviously, we have plenty of AI discussions.
4
SPEAKER_01
And you know, I myself have plenty of ideas.
4
SPEAKER_01
But I consider Ilya one of the most prolific AI scientists in the entire world.
4
SPEAKER_01
And I think that I realized that maybe my super skill is being able to bring people to collaborate together, that I have some level of empathy that is unique in AI world.
4
SPEAKER_01
And that might come, you know, from either meditation, psychedelics, or let s say I read just hundreds of books on this topic.
4
SPEAKER_01
So, and I also went through a journey of, you know, I develop all sorts of algorithms.
4
SPEAKER_01
So I think that
4
SPEAKER_01
maybe I can, that s my superhuman skill.
4
SPEAKER_01
Ilya is one of the best AI scientists, but then I m pretty good in assembling teams.
4
SPEAKER_01
And I m also not holding to people, like I m growing people and then people become managers at OpenAI.
4
SPEAKER_01
I grew many of them, like research managers.
4
SPEAKER_00
So you find you find places where you re excellent and he finds like his his deep scientific insights is where he is.
4
SPEAKER_00
And you find ways you can the puzzle pieces fit together.
4
SPEAKER_01
Correct.
4
SPEAKER_01
Like, you know, ultimately, for instance, let s say Ilya, he doesn t manage people.
4
SPEAKER_01
That s not what he likes or so.
4
SPEAKER_01
I like I like hanging out with people.
4
SPEAKER_01
By default, I m an extrovert and I care about people.
4
SPEAKER_00
Oh, interesting.
4
SPEAKER_00
OK.
4
SPEAKER_00
OK, cool.
4
SPEAKER_00
So that fits perfectly together.
4
SPEAKER_00
But I mean, I also just like your intuition about various problems in machine learning.
4
SPEAKER_00
He s definitely one I really enjoy.
4
SPEAKER_00
I remember talking to him about something I was struggling with, which is coming up with a good model for pedestrians, for human beings across the street in the context of autonomous vehicles.
4
SPEAKER_00
And he immediately started to formulate a framework within which you can evolve a model for pedestrians, like through self-play, all that kind of mechanisms.
4
SPEAKER_00
The depth of thought on a particular problem, especially problems he doesn t know anything about, is fascinating to watch.
4
SPEAKER_00
It makes you realize, like,
4
SPEAKER_00
Yeah, the limits that the human intellect might be limitless.
4
SPEAKER_00
Or it s just impressive to see a descendant of ape come up with clever ideas.
4
SPEAKER_01
Yeah, I mean, so even in the space of deep learning, when you look at various people, there are people now who invented some breakthroughs once, but there are very few people who did it multiple times.
4
SPEAKER_01
And you can think if someone invented it once, that might be just the sheer luck.
4
SPEAKER_01
And if someone invented it multiple times, if the probability of inventing it once is one over a million, then probability of inventing it twice or three times would be one over a million squared or to the power of three, which would be just impossible.
4
SPEAKER_01
So it literally means that it s given.
4
SPEAKER_01
that it s not the luck.
4
SPEAKER_01
And Ilya is one of these few people who have a lot of these inventions in his arsenal.
4
SPEAKER_01
It also feels that, you know, for instance, if you think about folks like Gauss or Euler, you know, at first they read a lot of books and then they did thinking and then they figure out math.
4
SPEAKER_01
And that s how it feels with Ilya.
4
SPEAKER_01
At first he read stuff and then he spent his thinking cycles.
4
SPEAKER_00
That s a really good way to put it.
4
SPEAKER_00
When I talk to him, I see thinking.
4
SPEAKER_00
He s actually thinking.
4
SPEAKER_00
He makes me realize that there s deep thinking that the human mind can do.
4
SPEAKER_00
Most of us are not thinking deeply.
4
SPEAKER_00
like you really have to put a lot of effort to think deeply.
4
SPEAKER_00
Like I have to really put myself in a place where I think deeply about a problem.
4
SPEAKER_00
It takes a lot of effort.
4
SPEAKER_00
It s like an airplane taking off or something.
4
SPEAKER_00
You have to achieve deep focus.
4
SPEAKER_00
He s just, what is it?
4
SPEAKER_00
His brain is like a vertical takeoff in terms of airplane analogy.
4
SPEAKER_00
So it s interesting, but
4
SPEAKER_00
I mean, Cal Newport talks about this as ideas of deep work.
4
SPEAKER_00
It s, you know, most of us don t work much at all in terms of like, like deeply think about particular problems, whether it s a math, engineering, all that kind of stuff.
4
SPEAKER_00
You want to go to that place often.
4
SPEAKER_00
And that s real hard work.
4
SPEAKER_00
And some of us are better than others at that.
4
SPEAKER_01
So I think that the big piece has to do with actually even engineering your environment such that it s conducive to that.
4
SPEAKER_01
So see, both Ilya and I, on the frequent basis, we kind of disconnect ourselves from the world in order to be able to do extensive amount of thinking.
4
SPEAKER_00
Yes.
4
SPEAKER_01
So Ilya usually, he just leaves iPad at hand.
4
SPEAKER_01
He loves his iPad.
4
SPEAKER_01
And for me, I m even sometimes, you know, just going for a few days to a different location, to Airbnb, I m turning off my phone and there is no access to me.
4
SPEAKER_01
And that s extremely important for me to be able to actually just formulate new thoughts, to do deep work rather than to be reactive.
4
SPEAKER_01
And the older I am, the more of these random tasks are at hand.
4
SPEAKER_00
Before I go on to that thread, let me return to our friend GPT.
4
SPEAKER_00
Let me ask you another ridiculously big question.
4
SPEAKER_00
Can you give an overview of what GPT-3 is?
4
SPEAKER_00
Or like you say in your Twitter bio, GPT-N plus one.
4
SPEAKER_00
how it works, and why it works.
4
SPEAKER_01
So GPT-3 is a humongous neural network.
4
SPEAKER_01
Let s assume that we know what is neural network by the definition.
4
SPEAKER_01
And it is trained on the entire internet just to predict next word.
4
SPEAKER_01
So let s say it sees part of the article, and the only task that it has at hand is to say what would be the next word, and what would be the next word.
4
SPEAKER_01
And it becomes really exceptional at the task of figuring out what s the next word.
4
SPEAKER_01
So you might ask, why would this be an important task?
4
SPEAKER_01
Why would it be important to predict what s the next word?
4
SPEAKER_01
And it turns out that a lot of problems can be formulated as a text completion problem.
4
SPEAKER_01
So GPT is purely learning to complete the text.
4
SPEAKER_01
But you could imagine, for instance, if you are asking a question, who is the president of the United States, then GPT can give you an answer to it.
4
SPEAKER_01
It turns out that many more things can be formulated this way.
4
SPEAKER_01
You can format text in the way that you have sent us in English.
4
SPEAKER_01
you make it even look like some content of a website elsewhere, which would be teaching people how to translate things between languages.
4
SPEAKER_01
So it would be EN colon text in English, FR colon, and then you ask people, and then you ask model to continue.
4
SPEAKER_01
And it turns out that such a model is predicting translation from English to French.
4
SPEAKER_01
The crazy thing is that
4
SPEAKER_01
This model can be used for way more sophisticated tasks.
4
SPEAKER_01
So you can format text such that it looks like a conversation between two people.
4
SPEAKER_01
And that might be a conversation between you and Elon Musk.
4
SPEAKER_01
And because the model read all the text about Elon Musk,
4
SPEAKER_01
It will be able to predict Elon Musk words as it would be Elon Musk.
4
SPEAKER_01
It will speak about colonization of Mars, about sustainable future, and so on.
4
SPEAKER_01
And it s also possible to even give arbitrary personality to the model.
4
SPEAKER_01
You can say, here is a conversation with a friendly AI bot.
4
SPEAKER_01
and the model will complete the text as a friendly AI bot.
4
SPEAKER_00
So, I mean, how do I express how amazing this is?
4
SPEAKER_00
So just to clarify a conversation, generating a conversation between me and Elon Musk,
4
SPEAKER_00
it wouldn t just generate good examples of what Elon would say, it would get the syntax all correct.
4
SPEAKER_00
So like interview style, it would say like Elon Cohen and Lex Cohen.
4
SPEAKER_00
It s not just like inklings of semantic correctness.
4
SPEAKER_00
It s like the whole thing, grammatical,
4
SPEAKER_00
syntactic, semantic.
4
SPEAKER_00
It s just really, really impressive generalization.
4
SPEAKER_01
Yeah, I mean, I also want to, you know, provide some caveats.
4
SPEAKER_01
So it can generate few paragraphs of coherent text.
4
SPEAKER_01
But as you go to longer pieces, it actually goes off the rails.
4
SPEAKER_01
If you try to write a book, it won t work out this way.
4
SPEAKER_00
What way does it go off the rails, by the way?
4
SPEAKER_00
Is there interesting ways in which it goes off the rails?
4
SPEAKER_01
Like what falls apart first?
4
SPEAKER_01
So the model is trained on all the existing data that is out there, which means that it is not trained on its own mistakes.
4
SPEAKER_01
So for instance, if it would make a mistake, then I can give you an example.
4
SPEAKER_01
So let s say I have a conversation with a model pretending that is Elon Musk.
4
SPEAKER_01
And then I start putting some, I start actually making up things which are not factual.
4
SPEAKER_01
I would say- Sounds like Twitter.
4
SPEAKER_00
But I gotcha.
4
SPEAKER_00
Sorry.
4
SPEAKER_00
Yeah.
4
SPEAKER_01
OK.
4
SPEAKER_01
I don t know.
4
SPEAKER_01
I would say that Elon is my wife.
4
SPEAKER_01
And the model will just keep on carrying it on.
4
SPEAKER_01
As if it s true.
4
SPEAKER_01
Yes.
4
SPEAKER_01
And in some sense, if you would have a normal conversation with Elon, he would be what the fuck.
4
SPEAKER_00
Yeah.
4
SPEAKER_00
There would be some feedback.
4
SPEAKER_00
So the model is trained on things that humans have written, but through the generation process, there s no human in the loop feedback.
4
SPEAKER_00
That s fascinating.
4
SPEAKER_00
So it s magnifies, like the errors get magnified and magnified.
4
SPEAKER_00
And it s also interesting.
4
SPEAKER_00
I mean, first of all, humans have the same problem.
4
SPEAKER_00
It s just that we make fewer errors and magnify the errors slower.
4
SPEAKER_01
I think that actually what happens with humans is if you have a wrong belief about the world as a kid, then very quickly you will learn that it s not correct because you are grounded in reality and you are learning from your new experience.
4
SPEAKER_01
Yes.
4
SPEAKER_00
But do you think the model can correct itself too?
4
SPEAKER_00
Won t it through the power of the representation and so the absence of Elon Musk being your wife, information on the internet, won t it correct itself?
4
SPEAKER_01
There won t be examples like that.
4
SPEAKER_00
So the errors will be subtle at first.
4
SPEAKER_01
Subtle at first.
4
SPEAKER_01
And in some sense, you can also say that the data that is not out there is the data which would represent how the human learns.
4
SPEAKER_01
And maybe the model would be trained on such a data, then it would be better off.
4
SPEAKER_00
How intelligent is GPT-3, do you think?
4
SPEAKER_00
When you think about the nature of intelligence, it seems exceptionally impressive.
4
SPEAKER_00
But then if you think about the big AGI problem, is this footsteps along the way to AGI?
4
SPEAKER_01
So let s see.
4
SPEAKER_01
It seems that intelligence itself is multiple axis of it.
4
SPEAKER_01
And I would expect that the systems that we are building, they might end up being superhuman on some axis and subhuman on some other axis.
4
SPEAKER_01
It would be surprising to me on all axis simultaneously they would become superhuman.
4
SPEAKER_01
Of course, people ask this question, is GPT a spaceship that
4
SPEAKER_01
that would take us to the moon?
4
SPEAKER_01
Or are we building a ladder to heaven that we are just building bigger and bigger ladder?
4
SPEAKER_01
And we don t know, in some sense, which one of these two... Which one is better?
4
SPEAKER_00
I m trying to... I like Stairway to Heaven, it s a good song, so I m not exactly sure which one is better.
4
SPEAKER_00
But you re saying like the spaceship to the moon is actually effective?
4
SPEAKER_01
Correct.
4
SPEAKER_01
So people who criticize GPT, they say, you are guys just
4
SPEAKER_01
building a taller ladder, and it will never reach the moon.
4
SPEAKER_01
And at the moment, I would say the way I m thinking is, is like a scientific question.
4
SPEAKER_01
And I m also in heart, I m a builder creator.
4
SPEAKER_01
And like, I m thinking, let s try out, let s see how far it goes.
4
SPEAKER_01
And so far, we see constantly that there is a progress.
4
SPEAKER_01
Yeah.
4
SPEAKER_00
So do you think GPT-4
4
SPEAKER_00
GPT-5, GPT-N plus one, there ll be a phase shift, like a transition to a place where we ll be truly surprised.
4
SPEAKER_00
Then again, like GPT-3 is already truly surprising.
4
SPEAKER_00
The people that criticize GPT-3 as a ladder to heaven,
4
SPEAKER_00
I think too quickly get accustomed to how impressive it is that the prediction of the next word can achieve such depth of semantics, accuracy of syntax, grammar, and semantics.
4
SPEAKER_00
Do you think GPT-4 and 5 and 6 will continue to surprise us?
4
SPEAKER_01
I mean, definitely, there will be more impressive models.
4
SPEAKER_01
There is a question, of course, if there will be a phase shift.
4
SPEAKER_01
Also, even the way I m thinking about these models is that when we build these models, we see some level of the capabilities, but we don t even fully understand everything that the model can do.
4
SPEAKER_01
And actually, one of the best things to do is to allow other people to probe the model to even see what is possible.
4
SPEAKER_00
Hence, using GPT as an API and opening it up to the world.
4
SPEAKER_01
Yeah, I mean, so when I m thinking from perspective of
4
SPEAKER_01
because there are obviously various people that have concerns about AGI, including myself.
4
SPEAKER_01
And then when I m thinking from perspective, what s the strategy even to deploy these things to the world?
4
SPEAKER_01
The one strategy that I have seen many times working is the iterative deployment, that you deploy slightly better versions and you allow other people to criticize you.
4
SPEAKER_01
So you actually, or try it out, you see where are their fundamental issues,
4
SPEAKER_01
And it s almost you don t want to be in that situation that you are holding into a powerful system and there s like a huge overhang.
4
SPEAKER_01
Then you deploy it and it might have a random chaotic impact on the world.
4
SPEAKER_01
So you actually want to be in the situation that you are gradually deploying systems.
4
SPEAKER_00
I asked this question of Ilya.
4
SPEAKER_00
Let me ask you this question.
4
SPEAKER_00
I ve been reading a lot about Stalin and power.
4
SPEAKER_00
If you re in possession of a system that s like AGI, that s exceptionally powerful, do you think your character and integrity might become corrupted, like famously power corrupts and absolute power corrupts absolutely?
4
SPEAKER_01
So I believe that you want at some point to work toward distributing the power.
4
SPEAKER_01
I think that you want to be in the situation that actually AGI is not controlled by a small number of people, but essentially by a larger collective.
4
SPEAKER_00
So the thing is, that requires a George Washington style move.
4
SPEAKER_00
in the ascent to power there s always a a moment when somebody gets a lot of power and they have to have the integrity and uh... the moral compass to give away that power
4
SPEAKER_00
that humans have been good and bad throughout history at this particular step.
4
SPEAKER_00
And I wonder, I wonder we like blind ourselves in, for example, between nations, a race towards, yeah, AI race between nations.
4
SPEAKER_00
We might blind ourselves and justify to ourselves the development of AI without distributing the power because we wanna defend ourselves against China, against Russia, that kind of logic.
4
SPEAKER_00
And I wonder how we design governance mechanisms that prevent us from becoming power hungry and in the process destroying ourselves.
4
SPEAKER_01
So let s see.
4
SPEAKER_01
I have been thinking about this topic quite a bit, but I also want to admit that once again, I actually want to rely way more on Sam Altman on it.
4
SPEAKER_01
He wrote an excellent blog on how even to distribute wealth.
4
SPEAKER_01
And he proposed in his blog to tax equity of the companies rather than profit and to distribute it.
4
SPEAKER_01
And this is an example of
4
SPEAKER_01
Washington move.
4
SPEAKER_01
I guess I personally have insane trust in Sam.
4
SPEAKER_01
He already spent plenty of money running Universal Basic Income project that gives me, I guess, maybe some level of trust to him.
4
SPEAKER_01
But I also, I guess, love him as a friend.
4
SPEAKER_00
Yeah, I wonder because we re sort of summoning a new set of technologies.
4
SPEAKER_00
I wonder if we ll be cognizant.
5
SPEAKER_01
like you re describing the process at OpenAI, but it could also be at other places like in the US government, right?
5
SPEAKER_01
Both China and the US are now full steam ahead on autonomous weapons systems development.
5
SPEAKER_01
And that s really worrying to me because in the framework of something being a national security danger or a military danger, you can do a lot of pretty dark things.
5
SPEAKER_01
that blind our moral compass.
5
SPEAKER_01
And I think AI will be one of those things.
5
SPEAKER_01
In some sense, the mission and the work you re doing at OpenAI is like the counterbalance to that.
5
SPEAKER_01
So you want to have more OpenAI and less autonomous weapon systems.
5
SPEAKER_00
I like these statements, like to be clear, like this is interesting and I m thinking about it myself, but this is a place that I put my trust actually in Sam s hands because it s extremely hard for me to reason about it.
5
SPEAKER_00
Yeah.
5
SPEAKER_01
I mean, one important statement to make is it s good to think about this.
5
SPEAKER_00
Yeah, no question about it.
5
SPEAKER_00
No question.
5
SPEAKER_01
Even like low level, quote unquote, engineer.
5
SPEAKER_01
Like there s such a
5
SPEAKER_01
I remember I programmed a car, our RC car.
5
SPEAKER_01
They went really fast, like 30, 40 miles an hour.
5
SPEAKER_01
And I remember I was like sleep deprived.
5
SPEAKER_01
So I programmed it pretty crappily and it like the code froze.
5
SPEAKER_01
So it s doing some basic computer vision and it s going around on track, but it s going full speed.
5
SPEAKER_01
And there was a bug in the code that the car just went
5
SPEAKER_01
it didn t turn, it went straight full speed and smashed into the wall.
5
SPEAKER_01
And I remember thinking the seriousness with which you need to approach the design of artificial intelligence systems and the programming of artificial intelligence systems is high, because the consequences are high.
5
SPEAKER_01
Like that little car smashing into the wall,
5
SPEAKER_01
For some reason, I immediately thought of like an algorithm that controls nuclear weapons having the same kind of bug.
5
SPEAKER_01
And so like the lowest level engineer and the CEO of a company all need to have the seriousness in approaching this problem and thinking about the worst case consequences.
5
SPEAKER_00
So I think that is true.
5
SPEAKER_00
I mean, what I also recognize in myself and others even asking this question is that it evokes a lot of fear and fear itself ends up being actually quite debilitating.
5
SPEAKER_00
The place where I arrived at the moment
5
SPEAKER_00
might sound cheesy or so, but it s almost to build things out of love rather than fear.
5
SPEAKER_00
Yeah.
5
SPEAKER_00
Like a focus on how I can, you know, maximize the value, how the systems that I m building might be useful.
5
SPEAKER_00
I m not saying that the fear doesn t exist out there and like it totally makes sense to minimize it.
5
SPEAKER_00
But I don t want to be working because I m scared.
5
SPEAKER_00
I want to be working out of passion, out of curiosity, out of looking forward for the positive future.
5
SPEAKER_01
With the definition of love arising from rigorous practice of empathy.
5
SPEAKER_01
So not just like your own conception of what is good for the world, but always listening to others.
5
SPEAKER_00
Correct, like at the loft where I m considering reward functions of others.
5
SPEAKER_01
Limit to infinity is like a sum, like 1 to n, where n is 7 billion or whatever it is.
5
SPEAKER_00
Not projecting my reward functions on others.
5
SPEAKER_01
Yeah, exactly.
5
SPEAKER_01
OK.
5
SPEAKER_01
Can we just take a step back to something else super cool, which is OpenAI Codecs?
5
SPEAKER_01
Can you give an overview of what OpenAI Codecs and GitHub Copilot is, how it works, and why the hell it works so well?
5
SPEAKER_00
So with GPT-3, we noticed that the system train on all the language out there started having some rudimentary coding capabilities.
5
SPEAKER_00
So we re able to ask it to implement addition function between two numbers.
5
SPEAKER_00
And indeed, it can write Python or JavaScript code for that.
5
SPEAKER_00
And then we thought we might as well just go full steam ahead and try to create a system that is actually good at what we are doing every day ourselves, which is programming.
5
SPEAKER_00
We optimize models for proficiency in coding.
5
SPEAKER_00
We actually even created models that both have a comprehension of language and code.
5
SPEAKER_00
And Codex is API for these models.
5
SPEAKER_01
So it s first pre-trained on language, and then I don t know if you can say fine-tuned, because there s a lot of code.
5
SPEAKER_01
But it s language and code.
5
SPEAKER_00
It s language and code.
5
SPEAKER_00
It s also optimized for various things like, let s say, low latency and so on.
5
SPEAKER_00
Codex is the API that s similar to GPT-3.
5
SPEAKER_00
We expect that there will be proliferation of the potential products that can use coding capabilities.
5
SPEAKER_00
And I can speak about it in a second.
5
SPEAKER_00
Copilot is the first product developed by GitHub.
5
SPEAKER_00
So as we re building models, we wanted to make sure that these models are useful.
5
SPEAKER_00
And we worked together with GitHub on building the first product.
5
SPEAKER_00
Copilot is actually, as you code, it suggests you code completions.
5
SPEAKER_00
And we have seen in the past, there are various tools that can suggest how to, like a few characters of the code or a line of code.
5
SPEAKER_00
The thing about Copilot is it can generate 10 lines of code.
5
SPEAKER_00
It s often the way how it works is you often write in the comment what you want to happen because
5
SPEAKER_00
People in comments, they describe what happens next.
5
SPEAKER_00
So these days, when I code, instead of going to Google to search for the appropriate code to solve my problem, I say, oh, for this array, could you smooth it?
5
SPEAKER_00
And then it imports some appropriate libraries and say it uses NumPy convolution or so that I was not even aware that exists.
5
SPEAKER_00
And it does the appropriate thing.
5
SPEAKER_01
So you write a comment, maybe the header of a function, and it completes the function.
5
SPEAKER_01
Of course, you don t know what is the space of all the possible small programs it can generate.
5
SPEAKER_01
What are the failure cases?
5
SPEAKER_01
How many edge cases?
5
SPEAKER_01
How many subtle errors there are?
5
SPEAKER_01
How many big errors there are?
5
SPEAKER_01
It s hard to know, but the fact that it works at all in a large number of cases is incredible.
5
SPEAKER_01
It s a kind of search engine into code that s been written on the internet.
5
SPEAKER_00
Correct.
5
SPEAKER_00
So for instance, when you search things online, then usually you get to some particular case.
5
SPEAKER_00
Like if you go to Stack Overflow, people describe that one particular situation.
5
SPEAKER_00
And then they seek for a solution.
5
SPEAKER_00
But in case of Copilot, it s aware of your entire context.
5
SPEAKER_00
And in context, it s, oh, these are the libraries that they are using.
5
SPEAKER_00
That s the set of the variables that is initialized.
5
SPEAKER_00
And on the spot, it can actually tell you what to do.
5
SPEAKER_00
So the interesting thing is, and we think that the Copilot is one possible product using Codex, but there is a place for many more.
5
SPEAKER_00
So internally, we tried out, you know, to create other fun products.
5
SPEAKER_00
So it turns out that a lot of tools out there, let s say Google Calendar or Microsoft Word or so, they all have internal API to build plugins around them.
5
SPEAKER_00
So there is a way, in a sophisticated way, to control Calendar or Microsoft Word.
5
SPEAKER_00
Today, if you want more complicated behaviors from these programs, you have to add a new button for every behavior.
5
SPEAKER_00
But it is possible to use codecs and tell, for instance, to Calendar, could you schedule an appointment with Lex next week after 2 p.m.?
5
SPEAKER_00
And it writes corresponding piece of code.
5
SPEAKER_00
And that s the thing that actually you want.
5
SPEAKER_00
So interesting.
5
SPEAKER_01
So what you figure out is there s a lot of programs with which you can interact through code.
5
SPEAKER_01
And so there, you can generate that code from natural language.
5
SPEAKER_01
That s fascinating.
5
SPEAKER_00
And that s somewhat like also closest to what was the promise of Siri or Alexa.
5
SPEAKER_00
So previously, all these behaviors, they were hardcoded.
5
SPEAKER_00
And it seems that Codex
5
SPEAKER_00
on the fly can pick up the API of, let s say, given software.
5
SPEAKER_00
And then it can turn language into use of this API.
5
SPEAKER_01
So without hard coding, it can translate to machine language.
5
SPEAKER_01
Correct.
5
SPEAKER_01
So for example, this would be really exciting for me, like for Adobe products like Photoshop, which I think ActionScript, I think there s a scripting language that communicates with them.
5
SPEAKER_01
Same with Premiere.
5
SPEAKER_00
And you could imagine that that allows even to do coding by voice on your phone.
5
SPEAKER_00
So for instance, in the past, as of today, I m not editing Word documents on my phone, because it s just the keyboard is too small.
5
SPEAKER_00
But if I would be able to tell to my phone, make the header large, then move the paragraphs around, and that s actually what I want.
5
SPEAKER_00
So I can tell you one more cool thing, or even how I m thinking about codecs.
5
SPEAKER_00
So if you look actually at the evolution of computers, we started with very primitive interfaces, which is a punch card.
5
SPEAKER_00
And punch card, essentially, you make holes in the plastic card to indicate zeros and ones.
5
SPEAKER_00
And during that time, there was a small number of specialists who were able to use computers.
5
SPEAKER_00
And by the way, people even suspected that there is no need for many more people to use computers.
5
SPEAKER_00
But then we moved from punch cards to, at first, assembly and C. And these programming languages, they were slightly higher level.
5
SPEAKER_00
They allowed many more people to code.
5
SPEAKER_00
And they also led to more of a proliferation of technology.
5
SPEAKER_00
And, you know, further on, there was a jump to, say, from C++ to Java and Python.
5
SPEAKER_00
And every time it has happened, more people are able to code and we build more technology.
5
SPEAKER_00
And it s even, you know,
5
SPEAKER_00
hard to imagine now if someone will tell you that you should write code in assembly instead of, let s say, Python or Java or JavaScript.
5
SPEAKER_00
And CodeX is yet another step toward kind of bringing computers closer to humans, such that you communicate with a computer with your own language rather than with a specialized language.
5
SPEAKER_00
And I think that it will lead to an increase of number of people who can code.
5
SPEAKER_01
Yeah, and the kind of technologies that those people will create, it s innumerable.
5
SPEAKER_01
It could be a huge number of technologies we re not predicting at all, because that s less and less requirement of having a technical mind, a programming mind.
5
SPEAKER_01
You re not opening it to the world of
5
SPEAKER_01
other kinds of minds, creative minds, artistic minds, all that kind of stuff.
5
SPEAKER_00
I would like, for instance, biologists who work on DNA to be able to program and not to need to spend a lot of time learning it.
5
SPEAKER_00
And I believe that s a good thing to the world.
5
SPEAKER_00
And I would actually add, I would add, so at the moment, I m managing codecs team and also language team.
5
SPEAKER_00
And I believe that there is like a plenty of brilliant people out there and they should apply.
5
SPEAKER_00
Oh, okay.
5
SPEAKER_01
Yeah.
5
SPEAKER_01
Awesome.
5
SPEAKER_01
So what s the language in the codex?
5
SPEAKER_01
So those are kind of their overlapping teams.
5
SPEAKER_01
So it s like GPT, the raw language, and then the codex is like applied to programming.
5
SPEAKER_00
Correct.
5
SPEAKER_00
And they are quite intertwined.
5
SPEAKER_00
There are many more teams involved making these models extremely efficient and deployable.
5
SPEAKER_00
For instance, there are people who are working to make our data centers amazing, or there are people who work on putting these models into production, or even pushing it at the very limit of the scale.
5
SPEAKER_00
So all aspects from the infrastructure to the actual machine learning.
5
SPEAKER_00
So I m just saying there are multiple teams.
5
SPEAKER_00
While the team working on codecs and language, I guess I m directly managing them.
5
SPEAKER_00
I would love to hire more.
5
SPEAKER_01
Yeah, if you re interested in machine learning,
5
SPEAKER_01
this is probably one of the most exciting problems and like systems to be working on.
5
SPEAKER_01
Because it s actually, it s pretty cool.
5
SPEAKER_01
Like what the program synthesis, the generating of programs is very interesting, very interesting problem that has echoes of reasoning and intelligence in it.
5
SPEAKER_01
And I think there s a lot of fundamental questions that you might be able to sneak.
5
SPEAKER_00
uh sneak up to by generating programs yeah the one more exciting thing about the programs is that so i said that the um you know the in case of language that one of the travels is even evaluating language so when the things are made up you you need somehow
5
SPEAKER_00
either a human to say that this doesn t make sense.
5
SPEAKER_00
Or so in case of program, there is this one extra lever that we can actually execute programs and see what they evaluate to.
5
SPEAKER_00
So that process might be somewhat more automated in order to improve the qualities of generations.
5
SPEAKER_00
Oh, that s fascinating.
5
SPEAKER_01
So like the, wow, that s really interesting.
5
SPEAKER_00
So for the language, the simulation to actually execute it, that s a human mind.
5
SPEAKER_00
For programs, there is a computer on which you can evaluate it.
5
SPEAKER_01
Wow.
5
SPEAKER_01
That s a brilliant little insight that the thing compiles and runs.
5
SPEAKER_01
That s first.
5
SPEAKER_01
And second, you can evaluate, do automated unit testing.
5
SPEAKER_00
And in some sense, it seems to me that we ll be able to make a tremendous progress.
5
SPEAKER_00
We are in the paradigm that there is way more data.
5
SPEAKER_00
There is like a transcription of millions of software engineers.
5
SPEAKER_00
Yeah.
5
SPEAKER_01
Yeah.
5
SPEAKER_01
So I mean, you just mean because I was going to ask you about reliability.
5
SPEAKER_01
The thing about programs is you don t know if they re going to
5
SPEAKER_01
Like a program that s controlling a nuclear power plant has to be very reliable.
5
SPEAKER_00
So I wouldn t start with controlling nuclear power plant.
5
SPEAKER_00
Maybe one day, but that s not actually, that s not on the current roadmap.
5
SPEAKER_00
That s not the step one.
5
SPEAKER_01
You know, it s the Russian thing.
5
SPEAKER_01
You just want to go to the most powerful, destructive thing right away, run by JavaScript.
5
SPEAKER_01
But I got you.
5
SPEAKER_01
So this is a lower impact.
5
SPEAKER_01
But nevertheless, what you make me realize, it is possible to achieve some levels of reliability by doing testing.
5
SPEAKER_00
And you could imagine that maybe there are ways for a model to write event code for testing itself and so on.
5
SPEAKER_00
And there exist ways to create the feedback loops that the model could keep on improving.
5
SPEAKER_01
by writing programs that generate tests.
5
SPEAKER_01
For the instance.
5
SPEAKER_01
And that s how we get consciousness, because it s metacompression.
5
SPEAKER_01
That s what you re going to write.
5
SPEAKER_01
That s the comment.
5
SPEAKER_01
That s the prompt that generates consciousness.
5
SPEAKER_01
Compressor of compressors.
5
SPEAKER_01
You just write that.
5
SPEAKER_01
Do you think the code that generates consciousness would be simple?
5
SPEAKER_00
So let s see.
5
SPEAKER_00
I mean, ultimately, the core idea behind will be simple, but there will be also a decent amount of engineering involved.
5
SPEAKER_00
In some sense, it seems that spreading these models on many machines, it s not that trivial.
5
SPEAKER_00
And we find all sorts of innovations that make our models more efficient.
5
SPEAKER_00
I believe that first models
5
SPEAKER_00
that I guess are conscious or like a truly intelligent, they will have all sorts of tricks.
5
SPEAKER_01
But then again, there s a certain argument that maybe the tricks are temporary things.
5
SPEAKER_00
Yeah, they might be temporary things.
5
SPEAKER_00
And in some sense, it s also even important to know that even the cost of a trick.
5
SPEAKER_00
So sometimes people are eager to put the trick
5
SPEAKER_00
while forgetting that there is a cost of maintenance.
5
SPEAKER_00
Or like a long-term cost.
5
SPEAKER_00
Long-term cost, or maintenance, or maybe even flexibility of code to actually implement new ideas.
5
SPEAKER_00
So even if you have something that gives you 2x, but it requires 1,000 lines of code, I m not sure if it s actually worth it.
5
SPEAKER_00
So in some sense, if it s five lines of code and 2x, I would take it.
5
SPEAKER_00
And we see many of this, but also, you know, that requires some level of, I guess, lack of attachment to code that we are willing to remove it.
5
SPEAKER_00
Yeah.
5
SPEAKER_01
So you led the OpenAI Robotics team.
5
SPEAKER_01
Can you give an overview of the cool things you were able to accomplish?
5
SPEAKER_01
What are you most proud of?
5
SPEAKER_00
So when we started robotics, we knew that actually reinforcement learning works and it is possible to solve fairly complicated problems.
5
SPEAKER_00
Like for instance, AlphaGo is an evidence that it is possible to build superhuman Go players.
5
SPEAKER_00
Dota2 is an evidence that it s possible to
5
SPEAKER_00
build superhuman agents playing Dota.
5
SPEAKER_00
So I asked myself a question, you know, what about robots out there?
5
SPEAKER_00
Could we train machines to solve arbitrary tasks in the physical world?
5
SPEAKER_00
Our approach was, I guess, let s pick a complicated problem that if we would solve it, that means that we made some significant progress in the domain.
5
SPEAKER_00
And then we went after the problem.
5
SPEAKER_00
So we noticed that actually the robots out there, they are kind of, at the moment, optimized per task.
5
SPEAKER_00
So you can have a robot that it s like if you have a robot opening a bottle, it s very likely that the end factor is that bottle opener.
5
SPEAKER_00
And in some sense, that s a hack to be able to solve a task, which makes any task easier.
5
SPEAKER_00
And I asked myself, so what would be a robot that can actually solve many tasks?
5
SPEAKER_00
And we concluded that human hands have such a quality that indeed, you have five kinds
6
SPEAKER_01
tiny arms attached.
6
SPEAKER_01
Individually, they can manipulate a pretty broad spectrum of objects.
6
SPEAKER_01
So we went after a single hand, like trying to solve Rubik s Cube single-handed.
6
SPEAKER_01
We picked this task because we thought that there is no way to hard-code it.
6
SPEAKER_01
And also, we picked a robot on which it would be hard to hard-code it.
6
SPEAKER_01
And we went after the solution such that
6
SPEAKER_01
it could generalize to other problems.
6
SPEAKER_00
And just to clarify, it s one robotic hand solving the Rubik s Cube.
6
SPEAKER_00
The hard part is in the solution to the Rubik s Cube is the manipulation of having it not fall out of the hand, having it use the five baby arms to rotate different parts of the Rubik s Cube to achieve the solution.
6
SPEAKER_00
Correct.
6
SPEAKER_00
Yeah.
6
SPEAKER_00
So what was the hardest part about that?
6
SPEAKER_00
What was the approach taken there?
6
SPEAKER_01
What are you most proud of?
6
SPEAKER_01
Obviously, we have like a strong belief in reinforcement learning.
6
SPEAKER_01
And, you know, one path it is to do reinforcement learning in the real world.
6
SPEAKER_01
other path is to the simulation.
6
SPEAKER_01
In some sense, the tricky part about the real world is, at the moment, our models, they require a lot of data.
6
SPEAKER_01
There is essentially no data.
6
SPEAKER_01
And we decided to go through the path of the simulation.
6
SPEAKER_01
And in simulation, you can have infinite amount of data.
6
SPEAKER_01
The tricky part is the fidelity of the simulation.
6
SPEAKER_01
And also, can you, in simulation, represent everything that you represent otherwise in the real world?
6
SPEAKER_01
And it turned out that because there is lack of fidelity, it is possible to... What we arrived at is training a model that doesn t solve one simulation, but it actually solves the entire range of simulations, which vary in terms of what s exactly the friction of the cube or weight or so.
6
SPEAKER_01
And the single AI that can solve all of them ends up working well with the reality.
6
SPEAKER_00
How do you generate the different simulations?
6
SPEAKER_01
So, you know, there s plenty of parameters out there.
6
SPEAKER_01
We just pick them randomly.
6
SPEAKER_01
And in simulation model just goes for thousands of years and keeps on solving Rubik s Cube in each of them.
6
SPEAKER_01
And the thing is that neural network that we used, it has a memory.
6
SPEAKER_01
And as it presses, for instance, the side of the
6
SPEAKER_01
of the cube, it can sense, oh, that s actually this side was difficult to press.
6
SPEAKER_01
I should press it stronger.
6
SPEAKER_01
And throughout this process, it kind of learns even how to solve this particular instance of the Ruby cube by giving us
6
SPEAKER_01
It s kind of like, you know, sometimes when you go to a gym and after bench press, you try to lift the glass and you kind of forgot and your head goes like up right away because kind of you got used to maybe different weight and it takes a second to adjust.
6
SPEAKER_01
And this kind of a memory that model came through the process of interacting with the cube in the simulation.
6
SPEAKER_00
I appreciate you speaking to the audience with a bench press.
6
SPEAKER_00
All the bros in the audience probably working out right now.
6
SPEAKER_00
There s probably somebody listening to this actually doing bench press.
6
SPEAKER_00
So maybe put the bar down and pick up the water bottle and you ll know exactly what Czech is talking about.
6
SPEAKER_00
OK, so what
6
SPEAKER_00
What was the hardest part of getting the whole thing to work?
6
SPEAKER_01
So the hardest part is at the moment when it comes to physical work, when it comes to robots, they require maintenance.
6
SPEAKER_01
It s hard to replicate a million times.
6
SPEAKER_01
It s also it s hard to replay things exactly.
6
SPEAKER_01
I remember this situation that
6
SPEAKER_01
One guy at our company, he had like a model that performs way better than other models in solving Rubik s Cube.
6
SPEAKER_01
And, you know, we kind of didn t know what s going on.
6
SPEAKER_01
Why is that?
6
SPEAKER_01
And
6
SPEAKER_01
It turned out that, you know, he was running it from his laptop that had better CPU or maybe local GPU as well.
6
SPEAKER_01
And because of that, there was less of a latency and the model was the same.
6
SPEAKER_01
And that actually made solving Rubik s Cube more reliable.
6
SPEAKER_01
So in some sense, there might be some saddlebags like that when it comes to running things in the real world.
6
SPEAKER_01
Even hinting on that,
6
SPEAKER_01
You could imagine that the initial models, you would like to have models which are insanely huge neural networks.
6
SPEAKER_01
And you would like to give them even more time for thinking.
6
SPEAKER_01
And when you have these real-time systems, then you might be constrained actually by the amount of latency.
6
SPEAKER_01
And ultimately, I would like to build a system that it is worth for you to wait five minutes because it gives you the answer that you are willing to wait for five minutes.
6
SPEAKER_01
So latency is a very unpleasant constraint under which to operate.
6
SPEAKER_01
Correct.
6
SPEAKER_01
And also there is actually one more thing which is tricky about robots.
6
SPEAKER_01
There is actually not much data.
6
SPEAKER_01
So the data that I m speaking about would be a data of first person experience from the robot.
6
SPEAKER_01
and like gigabytes of data like that.
6
SPEAKER_01
If we would have gigabytes of data like that of robots solving various problems, it would be very easy to make a progress on robotics.
6
SPEAKER_01
And you can see that in case of text or code, there is a lot of data, like a first-person perspective data on writing code.
6
SPEAKER_00
Yeah, so you had this, you mentioned this really interesting idea that if you were to build like a successful robotics company, so OpenAI s mission is much bigger than robotics.
6
SPEAKER_00
This is one of the, one of the things you ve worked on.
6
SPEAKER_00
But if it was a robotics company, that you wouldn t so quickly dismiss supervised learning.
6
SPEAKER_00
Correct, that you would build a robot that was perhaps an empty shell, like dumb, and it would operate under teleoperation.
6
SPEAKER_00
So you would invest, that s just one way to do it, invest in direct human control of the robots as it s learning.
6
SPEAKER_00
And over time, add more and more automation.
6
SPEAKER_01
That s correct.
6
SPEAKER_01
So let s say that s how I would build a robotics company today.
6
SPEAKER_01
If I would be building a robotics company, which is, you know, spend $10 million or so recording human trajectories, controlling a robot.
6
SPEAKER_00
After you find a thing that the robot should be doing that there s a market fit for, like you can make a lot of money with that product.
6
SPEAKER_00
Correct.
6
SPEAKER_01
Correct.
6
SPEAKER_00
Yeah.
6
SPEAKER_01
So I would record data and then I would essentially train supervised learning model on it.
6
SPEAKER_01
That might be the path today.
6
SPEAKER_01
Long term, I think that actually what is needed is to train powerful models over video.
6
SPEAKER_01
So you have seen maybe models that can generate images like DALI.
6
SPEAKER_01
And people are looking into models generating videos.
6
SPEAKER_01
There are like various algorithmic questions even how to do it.
6
SPEAKER_01
And it s unclear if there is enough compute for this purpose.
6
SPEAKER_01
But I suspect that the models that which would have
6
SPEAKER_01
Level of understanding of video, same as GPT has a level of understanding of text.
6
SPEAKER_01
could be used to train robots to solve tasks.
6
SPEAKER_01
They would have a lot of common sense.
6
SPEAKER_00
If one day, I m pretty sure one day, there will be a robotics company, by robotics company I mean the primary source of income is from robots, that is worth over $1 trillion.
6
SPEAKER_00
What do you think that company will do?
6
SPEAKER_01
I think sell driving cars, no?
6
SPEAKER_00
It s interesting because my mind went to personal robotics, robots in the home.
6
SPEAKER_00
It seems like there s much more market opportunity there.
6
SPEAKER_00
I think it s very difficult to achieve.
6
SPEAKER_00
I mean, this might speak to something important, which is I understand self-driving much better than I understand robotics in the home.
6
SPEAKER_00
So I understand how difficult it is to actually solve self-driving.
6
SPEAKER_00
to a level, not just the actual computer vision and the control problem and just the basic problems of driving, but creating a product.
6
SPEAKER_00
that would undeniably be, that would cost less money, like it would save you a lot of money, like orders of magnitude less money, that could replace Uber drivers, for example.
6
SPEAKER_00
So car sharing that s autonomous, that creates a similar or better experience in terms of how quickly you get from A to B, or just whatever, the pleasantness of the experience, the efficiency of the experience, the value of the experience, and at the same time, the car itself costs cheaper.
6
SPEAKER_00
I think that s very difficult to achieve.
6
SPEAKER_00
I think there s a lot more low-hanging fruit in the home.
6
SPEAKER_01
That could be.
6
SPEAKER_01
I also want to give you a perspective on how challenging it would be at home or maybe kind of depends on the exact problem that you d be solving.
6
SPEAKER_01
If we re speaking about these robotic arms and hands, these things, they cost tens of thousands of dollars or maybe 100K.
6
SPEAKER_01
You know, maybe obviously, maybe there would be economy of scale, these things would be cheaper.
6
SPEAKER_01
But actually for any household to buy it, the price would have to go down to maybe a thousand bucks.
6
SPEAKER_00
Yeah, I personally think that, so self-driving car provides a clear service.
6
SPEAKER_00
I don t think robots in the home, there ll be a trillion dollar company will just be all about service.
6
SPEAKER_00
Meaning it will not necessarily be about like a robotic arm that helps you, I don t know, open a bottle or wash the dishes or any of that kind of stuff.
6
SPEAKER_00
It has to be able to take care of that whole, the therapist thing you mentioned.
6
SPEAKER_00
I think that s, of course, there s a line between what is a robot and what is not.
6
SPEAKER_00
Like, does it really need a body?
6
SPEAKER_00
But, you know, some AI system with some embodiment, I think.
6
SPEAKER_01
So the tricky part when you think actually what s the difficult part is
6
SPEAKER_01
And when the robot has like when there is a diversity of the environment with which the robot has to interact, that becomes hard.
6
SPEAKER_01
So, you know, on one spectrum, you have industrial robots as they are doing over and over the same thing.
6
SPEAKER_01
It is possible to some extent to prescribe the movements.
6
SPEAKER_01
And with very small amount of intelligence, the movement can be repeated millions of times.
6
SPEAKER_01
There are also, you know, various pieces of industrial robots where it becomes harder and harder.
6
SPEAKER_01
Like, for instance, in case of Tesla, it might be a matter of putting a rack inside of a car.
6
SPEAKER_01
And, you know, because the rack kind of moves around, it s not that easy.
6
SPEAKER_01
It s not exactly the same every time.
6
SPEAKER_01
It ends up being the case that you need actually humans to do it.
6
SPEAKER_01
while, you know, welding cars together, it s a very repetitive process.
6
SPEAKER_01
Then in case of self-driving itself, the difficulty has to do with the diversity of the environment, but still the car itself, the problem that we are solving is
6
SPEAKER_01
you try to avoid even interacting with things.
6
SPEAKER_01
You are not touching anything around because touching itself is hard.
6
SPEAKER_01
And then if you would have in the home robot that, you know, has to touch things and like if these things, they change the shape, if there is a huge variety of things to be touched, then that s difficult.
6
SPEAKER_01
If you are speaking about the robot, which there is, you know, head that is smiling in some way with cameras that it doesn t, you know, touch things, that s relatively simple.
6
SPEAKER_00
OK.
6
SPEAKER_00
So to both agree and to push back.
6
SPEAKER_00
So you re referring to touch like soft robotics, like the actual touch.
6
SPEAKER_00
But I would argue that you could formulate just basic interaction between like non-contact interaction is also a kind of touch.
6
SPEAKER_00
And that might be very difficult to solve.
6
SPEAKER_00
That s the basic, not disagreement, but that s the basic open question to me with self-driving cars and disagreement with Elon, which is how much interaction is required to solve self-driving cars?
6
SPEAKER_00
How much touch is required?
6
SPEAKER_00
You said that in your intuition, touch is not required.
6
SPEAKER_00
In my intuition, to create a product that s compelling to use, you re going to have to interact with pedestrians, not just avoid pedestrians, but interact with them.
6
SPEAKER_00
When we drive around in major cities, we re constantly threatening everybody s life with our movements.
6
SPEAKER_00
And that s how they respect us.
6
SPEAKER_00
There s a game theory going on with pedestrians.
6
SPEAKER_00
And I m afraid you can t just formulate autonomous driving as a collision avoidance problem.
6
SPEAKER_01
So I think it goes beyond.
6
SPEAKER_01
Collision avoidance is the first order approximation.
6
SPEAKER_01
But then, at least in case of Tesla, they are gathering data from people driving their cars.
6
SPEAKER_01
And I believe that s an example of supervised learning data that they can train their models on.
6
SPEAKER_01
And they are doing it, which can give a model this another level of behavior that is needed to actually interact with the real world.
6
SPEAKER_00
Yeah, it s interesting how much data is required to achieve that.
6
SPEAKER_00
what do you think of the whole Tesla autopilot approach?
6
SPEAKER_00
The computer vision based approach with multiple cameras and as a data engine, it s a multi-task, multi-headed neural network.
6
SPEAKER_00
And it s this fascinating process of similar to what you re talking about with the robotics approach, which is, you know, you deploy neural network and then there s humans that use it and then it runs into trouble in a bunch of places and that stuff is sent back.
6
SPEAKER_00
So like,
6
SPEAKER_00
the deployment discovers a bunch of edge cases, and those edge cases are sent back for supervised annotation, thereby improving the neural network.
6
SPEAKER_00
And that s deployed again, it goes over and over until the network becomes really good at the task of driving, becomes safer and safer.
6
SPEAKER_00
What do you think of that kind of approach to robotics?
6
SPEAKER_01
I believe that s the way to go.
6
SPEAKER_01
So in some sense, even when I was speaking about, you know, collecting trajectories from humans, that s like a first step.
6
SPEAKER_01
And then you deploy the system and then you have humans revising all the issues.
6
SPEAKER_01
And in some sense, like this approach converges to system that doesn t make mistakes, because for the cases where there are mistakes, you got their data, how to fix them, and the system will keep on improving.
6
SPEAKER_00
So there s a very, to me, difficult question of how hard that, you know, how long that converging takes, how hard it is.
6
SPEAKER_00
The other aspect of autonomous vehicles, this probably applies to certain robotics applications, is society, right?
6
SPEAKER_00
They put
6
SPEAKER_00
As the quality of the system converges, so one is a human factors perspective of psychology of humans being able to supervise those, even with teleoperation, those robots.
6
SPEAKER_00
And the other is society willing to accept robots.
6
SPEAKER_00
Currently, society is much harsher on self-driving cars than it is on human-driven cars.
6
SPEAKER_00
in terms of the expectation of safety.
6
SPEAKER_00
So the bar is set much higher than for humans.
6
SPEAKER_00
And so if there s a death in an autonomous vehicle, that s seen as much more.
6
SPEAKER_00
much more dramatic than a death in a human-driven vehicle.
6
SPEAKER_00
Part of the success of deployment of robots is figuring out how to make robots part of society, both on just the human side, on the media journalist side, and also on the policy government side.
6
SPEAKER_00
And that seems to be, maybe you can put that into the objective function to optimize.
6
SPEAKER_00
But that is definitely a
6
SPEAKER_00
a tricky one and I wonder if that is actually the trickiest part for self-driving cars or any system that s safety critical.
6
SPEAKER_00
It s not the algorithm, it s the society accepting it.
6
SPEAKER_01
Yeah, I would say I believe that the part of the process of deployment is actually showing people that the given things can be trusted.
6
SPEAKER_01
Yeah.
6
SPEAKER_01
And, you know, trust is also like a glass that is actually really easy to crack it.
6
SPEAKER_01
Yeah.
6
SPEAKER_01
And damage it.
6
SPEAKER_01
And I think that s actually very common with
6
SPEAKER_01
with innovation, that there is some resistance toward it.
6
SPEAKER_01
And it s just a natural progression.
6
SPEAKER_01
So in some sense, people will have to keep on proving that, indeed, these systems are worth being used.
6
SPEAKER_01
And I would say, I also found out that often the best way to convince people is by letting them experience it.
6
SPEAKER_00
Yeah, absolutely.
6
SPEAKER_00
That s the case with Tesla Autopilot, for example.
6
SPEAKER_00
That s the case with, yeah, with basically robots in general.
6
SPEAKER_00
It s kind of funny to hear people talk about robots, like there s a lot of fear, even with like legged robots.
6
SPEAKER_00
But when they actually interact with them, there s joy.
6
SPEAKER_00
I love interacting with them.
6
SPEAKER_00
And the same with the car.
6
SPEAKER_00
With a robot, if it starts being useful, I think people immediately understand.
6
SPEAKER_00
And if the product is designed well, they fall in love.
6
SPEAKER_00
You re right.
6
SPEAKER_01
It s actually even similar when I m thinking about copilot, the GitHub copilot, there was a spectrum of responses that people had.
6
SPEAKER_01
And ultimately, the important piece was to let people try it out.
6
SPEAKER_01
And then many people just loved it.
6
SPEAKER_01
Especially like programmers.
6
SPEAKER_01
Yeah, programmers, but like some of them, you know, they came with a fear.
6
SPEAKER_01
But then you try it out and you think, actually, that s cool.
6
SPEAKER_01
And you can try to resist the same way as you could resist moving from punch cards to, let s say, C++ or so.
6
SPEAKER_00
And it s a little bit futile.
6
SPEAKER_00
So we talked about generation of program, generation of language, even self-supervised learning in the visual space for robotics and then reinforcement learning.
6
SPEAKER_00
What do you in like this whole beautiful spectrum of AI
7
SPEAKER_01
AI, do you think is a good benchmark, a good test to strive for to achieve intelligence?
7
SPEAKER_01
That s a strong test of intelligence?
7
SPEAKER_01
You know, it started with Alan Turing and the Turing test.
7
SPEAKER_01
Maybe you think natural language conversation is a good test?
7
SPEAKER_00
So, you know, it would be nice if, for instance, machine would be able to solve Riemann hypothesis in math.
7
SPEAKER_00
That would be, I think that would be very impressive.
7
SPEAKER_01
So theorem proving, is that to you, proving theorems is a good, oh, like one thing that the machine did, you would say, damn.
7
SPEAKER_00
Exactly.
7
SPEAKER_00
Okay.
7
SPEAKER_00
That would be quite impressive.
7
SPEAKER_00
I mean, the tricky part about the benchmarks is,
7
SPEAKER_00
You know, as we are getting closer with them, we have to invent new benchmarks.
7
SPEAKER_00
There is actually no ultimate benchmark out there.
7
SPEAKER_01
Yeah, see, my thought with the Riemann hypothesis would be the moment the machine proves it, we would say, OK, well, then the problem was easy.
7
SPEAKER_00
That s what happens.
7
SPEAKER_00
And I mean, in some sense, that s actually what happens over the years in AI, that like
7
SPEAKER_00
we get used to things very quickly.
7
SPEAKER_01
You know something, I talked to Rodney Brooks, I don t know if you know who that is, he called AlphaZero a homework problem.
7
SPEAKER_01
Because he was saying like, there s nothing special about it, it s not a big leap.
7
SPEAKER_01
And I didn t, well, he s coming from one of the aspects that we referred to is, he was part of the founding of iRobot, which deployed now tens of millions of robot in the home.
7
SPEAKER_01
So if you see robots,
7
SPEAKER_01
that are actually in the homes of people as the legitimate instantiation of artificial intelligence, then yes.
7
SPEAKER_01
Maybe an AI that plays a silly game like Go and chess is not a real accomplishment, but to me it s a fundamental leap.
7
SPEAKER_01
But I think we as humans then say, okay, well then that game of chess or Go wasn t that difficult compared to the thing that s currently unsolved.
7
SPEAKER_00
So my intuition is that from perspective of the evolution of
7
SPEAKER_00
you know, these AI systems will at first see the tremendous progress in digital space.
7
SPEAKER_00
And, you know, the main thing about digital space is also that you can everything is that there is a lot of recorded data.
7
SPEAKER_00
Plus, you can very rapidly deploy things to billions of people.
7
SPEAKER_00
Well, in case of physical space, the deployment part takes multiple years.
7
SPEAKER_00
You have to manufacture things and
7
SPEAKER_00
you know, delivering it to actual people, it s very hard.
7
SPEAKER_00
And so I m expecting that the first and that prices in digital space of goods, they would go, you know, down to the let s say marginal costs are zero.
7
SPEAKER_01
And also the question is how much of our life will be in digital?
7
SPEAKER_01
Because it seems like we re heading towards more and more of our lives being in the digital space.
7
SPEAKER_01
So like innovation in the physical space might become less and less significant.
7
SPEAKER_01
Like, why do you need to drive anywhere if most of your life is spent in virtual reality?
7
SPEAKER_00
I still would like, you know, to at least at the moment, my impression is that I would like to have a physical contact with other people.
7
SPEAKER_00
And that s very important to me.
7
SPEAKER_00
We don t have a way to replicate it in the computer.
7
SPEAKER_00
It might be the case that over the time it will change.
7
SPEAKER_01
like in 10 years from now, why not have like an arbitrary infinite number of people you can interact with?
7
SPEAKER_01
Some of them are real, some are not, with arbitrary characteristics that you can define based on your own preferences.
7
SPEAKER_00
I think that s maybe where we are heading and maybe I m resisting the future.
7
SPEAKER_00
Yeah.
7
SPEAKER_01
I m telling you, if I got to choose, if I could live in Elder Scrolls Skyrim versus the real world, I m not so sure I would stay with the real world.
7
SPEAKER_00
Yeah, I mean, the question is, so will VR be sufficient to get us there?
7
SPEAKER_00
Or do you need to, you know, plug electrodes in the brain?
7
SPEAKER_00
And it would be nice if these electrodes wouldn t be invasive.
7
SPEAKER_00
Yeah.
7
SPEAKER_01
Or at least like provably non-destructive.
7
SPEAKER_01
But in the digital space, do you think we ll be able to solve the Turing test, the spirit of the Turing test, which is, do you think we ll be able to achieve compelling natural language conversation between people, like have friends that are AI systems on the internet?
7
SPEAKER_00
I totally think it s doable.
7
SPEAKER_01
Do you think the current approach with GPT will take us there?
7
SPEAKER_00
Yes, so there is, you know, the part of at first learning all the content out there.
7
SPEAKER_00
And I think that still system should keep on learning as it speaks with you.
7
SPEAKER_00
Yeah.
7
SPEAKER_00
And I think that should work.
7
SPEAKER_00
The question is how exactly to do it.
7
SPEAKER_00
And, you know, obviously we have people at OpenAI asking these questions and kind of at first pre-training on all existing content is like a backbone and is a decent backbone.
7
SPEAKER_01
Do you think AI needs a body, connecting to our robotics question, to truly connect with humans?
7
SPEAKER_01
Or can most of the connection be in the digital space?
7
SPEAKER_00
So, let s see.
7
SPEAKER_00
We know that there are people who met each other online and they fell in love.
7
SPEAKER_00
Yeah.
7
SPEAKER_00
So it seems that it s conceivable to establish connection, which is purely through internet.
7
SPEAKER_00
Of course, it might be more compelling the more modalities you add.
7
SPEAKER_01
So it would be like you re proposing like a Tinder, but for AI.
7
SPEAKER_01
You like swipe right and left and half the systems are AI and the other is humans and you don t know which is which.
7
SPEAKER_00
That would be our formulation of Turing test.
7
SPEAKER_01
The moment AI is able to achieve more swipe right or left, whatever, the moment it s able to be more attractive than other humans, it passes the Turing test.
7
SPEAKER_00
Then you would pass the Turing test in attractiveness.
7
SPEAKER_01
Well, no, like attractiveness just to clarify.
7
SPEAKER_00
There will be conversation.
7
SPEAKER_01
Not just visual, right, right.
7
SPEAKER_01
It s also attractiveness with wit and humor and whatever makes conversations pleasant for humans.
7
SPEAKER_01
Okay, all right.
7
SPEAKER_01
So you re saying it s possible to achieve in a digital space.
7
SPEAKER_00
In some sense, I would almost ask the question, why wouldn t that be possible?
7
SPEAKER_01
Right.
7
SPEAKER_01
Well, I have this argument with my dad all the time.
7
SPEAKER_01
He thinks that touch and smell are really important.
7
SPEAKER_00
So they can be very important.
7
SPEAKER_00
And I m saying the initial systems, they won t have it.
7
SPEAKER_00
Still, I wouldn t, like, there are people being born without these senses.
7
SPEAKER_00
And, you know, I believe that they can still fall in love and have meaningful life.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
I wonder if it s possible to go close to all the way by just training on transcripts of conversations.
7
SPEAKER_01
Like, I wonder how far that takes us.
7
SPEAKER_00
So I think that actually still you want images, like I would like.
7
SPEAKER_00
So I don t have kids, but like I could imagine that having AI tutor, it has to see, you know, kids drawing some pictures on the paper.
7
SPEAKER_00
And facial expressions, all that kind of stuff.
7
SPEAKER_01
We use dogs and humans use their eyes.
7
SPEAKER_01
to communicate with each other.
7
SPEAKER_01
I think that s a really powerful mechanism of communication.
7
SPEAKER_01
Body language, too, that words are much lower bandwidth.
7
SPEAKER_00
And for body language, we still, you know, we can have a system that displays an image of its autofacial expression on the computer.
7
SPEAKER_00
It doesn t have to move, you know, mechanical pieces or so.
7
SPEAKER_00
So I think that there is like kind of a progression.
7
SPEAKER_00
You can imagine that text might be the simplest to tackle.
7
SPEAKER_00
But this is not a complete human experience at all.
7
SPEAKER_00
You expand it to, let s say, images both for input and output.
7
SPEAKER_00
And what you describe is actually the final, I guess, frontier, what makes us human, the fact that we can touch each other or smell or so.
7
SPEAKER_00
And it s the hardest from perspective of data and deployment.
7
SPEAKER_00
And I believe that these things might happen gradually.
7
SPEAKER_01
Are you excited by that possibility, this particular application of human to AI friendship and interaction?
7
SPEAKER_00
So let s see.
7
SPEAKER_01
Do you look forward to a world?
7
SPEAKER_01
You said you re living with a few folks and you re very close friends with them.
7
SPEAKER_01
Do you look forward to a day where one or two of those friends are AI systems?
7
SPEAKER_00
So if the system would be truly wishing me well, rather than being in the situation that it optimizes for my time to interact with the system,
7
SPEAKER_00
The line between those, it s a gray area.
7
SPEAKER_00
I think that s the distinction between love and possession.
7
SPEAKER_00
And these things, they might be often correlated for humans, but you might find that there are some friends with whom you haven t spoke for months.
7
SPEAKER_00
And then you pick up the phone as the time hasn t passed.
7
SPEAKER_00
They are not holding to you.
7
SPEAKER_00
And I wouldn t like to have AI system that, you know, it s trying to convince me to spend time with it.
7
SPEAKER_00
I would like the system to optimize for what I care about and help me in achieving my own goals.
7
SPEAKER_01
But there s some, I mean, I don t know.
7
SPEAKER_01
There s some manipulation, there s some possessiveness, there s some insecurities, there s fragility.
7
SPEAKER_01
All those things are necessary to form a close friendship over time, to go through some dark shit together, some bliss and happiness together.
7
SPEAKER_01
I feel like there s a lot of greedy, self-centered behavior within that process.
7
SPEAKER_00
My intuition, but I might be wrong, is that human-computer interaction doesn t have to go through a computer being greedy, possessive, and so on.
7
SPEAKER_00
It is possible to train systems, maybe, that they actually are, I guess, prompted or fine-tuned or so to truly optimize for what you care about.
7
SPEAKER_00
And you could imagine that the way how the process would look like is, at some point,
7
SPEAKER_00
We as humans, we look at the transcript of the conversation or like an entire interaction and we say, actually here there was more loving way to go about it.
7
SPEAKER_00
And we supervise system toward being more loving.
7
SPEAKER_00
Or maybe we train the system such that it has a reward function toward being more loving.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
Or maybe the possibility of the system being an asshole and manipulative and possessive every once in a while is a feature, not a bug.
7
SPEAKER_01
Because some of the happiness that we experience when two souls meet each other, when two humans meet each other, is a kind of break from the assholes in the world.
7
SPEAKER_01
And so you need assholes in AI as well, because it ll be like a breath of fresh air to discover an AI that the three previous AIs you had are too friendly.
7
SPEAKER_01
or cruel or whatever.
7
SPEAKER_01
It s like some kind of mix.
7
SPEAKER_01
And then this one is just right.
7
SPEAKER_01
But you need to experience the full spectrum.
7
SPEAKER_01
Like, I think you need to be able to engineer assholes.
7
SPEAKER_01
So, let s see.
7
SPEAKER_01
Because there s some level to us being appreciate, to appreciate the human experience.
7
SPEAKER_01
We need the dark.
7
SPEAKER_00
and the light.
7
SPEAKER_00
you know, beautiful, beautiful woman, and she had a crutch, okay?
7
SPEAKER_00
She had trouble walking on one leg.
7
SPEAKER_00
I asked her what has happened, and she said that five years ago she was in Maui, Hawaii, and she was eating a salad, and some snail fell into the salad, and apparently there are neurotoxic snails over there, and she got into coma for a year, okay?
7
SPEAKER_00
Oh, wow.
7
SPEAKER_00
And apparently there is a high chance of even just dying, but she was in the coma.
7
SPEAKER_00
At some point, she regained partially consciousness.
7
SPEAKER_00
She was able to hear people in the room.
7
SPEAKER_00
People behaved as if she wouldn t be there.
7
SPEAKER_00
You know, at some point she started being able to speak, but she was mumbling, like barely able to express herself.
7
SPEAKER_00
Then at some point she got into wheelchair.
7
SPEAKER_00
Then at some point she actually noticed that she can move her toe.
7
SPEAKER_00
And then she knew that she will be able to walk.
7
SPEAKER_00
And then, you know, that s where she was five years after.
7
SPEAKER_00
And she said that since then, she appreciates the fact that she can move her toe.
7
SPEAKER_00
And I was thinking, hmm, do I need to go through such experience to appreciate that I can move my toe?
7
SPEAKER_00
Wow, that s really good story and really deep example, yeah.
7
SPEAKER_00
And in some sense, it might be the case that we don t see light if we haven t went through the darkness, but I wouldn t say that we should.
7
SPEAKER_00
We shouldn t assume that that s the case.
7
SPEAKER_01
We may be able to engineer shortcuts.
7
SPEAKER_00
Yeah.
7
SPEAKER_00
Ilya had this, you know, belief that maybe one has to go for a week or six months to some challenging camp.
7
SPEAKER_00
Yeah.
7
SPEAKER_00
To just experience, you know, a lot of difficulties and then comes back and actually everything is bright.
7
SPEAKER_00
Everything is beautiful.
7
SPEAKER_01
I m with Ilya on this.
7
SPEAKER_01
It must be a Russian thing.
7
SPEAKER_01
Where are you from originally?
7
SPEAKER_00
I m Polish.
7
SPEAKER_01
Polish.
7
SPEAKER_01
Okay.
7
SPEAKER_01
I m tempted to say that explains a lot, but yeah, there s something about the Russian, the necessity of suffering.
7
SPEAKER_01
I believe suffering or rather struggle is necessary.
7
SPEAKER_00
I believe that struggle is necessary.
7
SPEAKER_00
I mean, in some sense, you even look at the story of any superhero, the movie, it s not that it was like everything goes easy, easy, easy, easy.
7
SPEAKER_01
I like how that s your ground truth is the story of superheroes.
7
SPEAKER_01
OK, you mentioned that you used to do research at night and go to bed at like six a.m.
7
SPEAKER_01
or seven a.m.. I still do that often.
7
SPEAKER_01
What sleep schedules have you tried to make for a productive and happy life?
7
SPEAKER_01
Is there some interesting wild sleeping patterns that you engaged that you found that works really well for you?
7
SPEAKER_00
I tried at some point decreasing number of hours of sleep, like gradually, like half an hour every few days less.
7
SPEAKER_00
You know, I was hoping to just save time.
7
SPEAKER_00
That clearly didn t work for me.
7
SPEAKER_00
Like at some point there s like a phase shift and I felt tired all the time.
7
SPEAKER_00
You know, there was a time that I used to work during the nights.
7
SPEAKER_00
The nice thing about nights is that no one disturbs you.
7
SPEAKER_00
And even I remember when I was meeting for the first time with Greg Brockman, his CTO and chairman of OpenAI, our meeting was scheduled to 5 p.m.
7
SPEAKER_00
and I overstepped for the meeting.
7
SPEAKER_01
Overslept for the meeting at 5 p.m.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
Now you sound like me.
7
SPEAKER_01
That s hilarious.
7
SPEAKER_01
OK.
7
SPEAKER_01
Yeah.
7
SPEAKER_00
And at the moment, in some sense, my sleeping schedule also has to do with the fact that I m interacting with people.
7
SPEAKER_00
I sleep without an alarm.
7
SPEAKER_01
So yeah, the team thing you mentioned, extrovert thing, because most humans operate during a certain set of hours.
7
SPEAKER_01
You re forced to then operate at the same set of hours.
7
SPEAKER_01
But I m not quite there yet.
7
SPEAKER_01
I found a lot of joy, just like you said, working through the night, because it s quiet, because the world doesn t disturb you.
7
SPEAKER_01
And there s some aspect, counter to everything you re saying, there s some joyful aspect to sleeping through the mess of the day.
7
SPEAKER_01
because people are having meetings and sending emails and there s drama meetings.
7
SPEAKER_00
I can sleep through all the meetings.
7
SPEAKER_00
You know, I have meetings every day and they prevent me from having sufficient amount of time for focused work.
7
SPEAKER_00
And.
7
SPEAKER_00
Then I modified my calendar and I said that I m out of office Wednesday, Thursday and Friday every day.
7
SPEAKER_00
And I m having meetings only Monday and Tuesday.
7
SPEAKER_00
And that vastly positively influenced my mood that I have literally like three days for fully focused work.
7
SPEAKER_01
So there s better solutions to this problem than staying awake all night.
7
SPEAKER_01
Okay, you ve been part of development of some of the greatest ideas in artificial intelligence.
7
SPEAKER_01
What would you say is your process for developing good novel ideas?
7
SPEAKER_00
You have to be aware that
7
SPEAKER_00
Clearly, there are many other brilliant people around.
7
SPEAKER_00
So you have to ask yourself a question, why the given idea, let s say, wasn t tried by someone else.
7
SPEAKER_00
And in some sense, it has to do with
7
SPEAKER_00
you know, kind of simple, it might sound simple, but like a thinking outside of the box.
7
SPEAKER_00
And what do I mean here?
7
SPEAKER_00
So for instance, for a while, people in academia, they assumed that you have a fixed data set.
7
SPEAKER_00
And then you optimize the algorithms in order to get the best performance.
7
SPEAKER_00
And that was so ingrained assumption that no one thought about training models on anti-internet.
7
SPEAKER_00
Or like that maybe some people thought about it, but it felt to many as unfair.
7
SPEAKER_00
And in some sense, that s almost like a, it s not my idea or so, but that s an example of breaking a typical assumption.
7
SPEAKER_00
So you want to be in the paradigm that you re breaking a typical assumption.
7
SPEAKER_01
In the context of the AI community, getting to pick your data set is cheating.
7
SPEAKER_00
Correct.
7
SPEAKER_00
And in some sense, so that was a that was assumption that many people had out there and then if you free yourself from assumptions, then they are likely to achieve something that others cannot do.
7
SPEAKER_00
And in some sense, if you are trying to do exactly the same things as others, it s very likely that you re going to have the same results.
7
SPEAKER_00
Yeah, I but there s also that kind of tension, which is
8
SPEAKER_02
asking yourself the question, why haven t others done this?
8
SPEAKER_02
Because, I mean, I get a lot of good ideas, but I think probably most of them suck when they meet reality.
8
SPEAKER_00
So actually, I think the other big piece is getting into habit of generating ideas, training your brain towards generating ideas, and not even suspending judgment of the ideas.
8
SPEAKER_00
So in some sense, I noticed myself that even if I m in the process of generating ideas, if I tell myself, oh, that was a bad idea,
8
SPEAKER_00
then that actually interrupts the process and I cannot generate more ideas because I m actually focused on the negative part, why it won t work.
8
SPEAKER_00
But I created also environment in the way that it s very easy for me to store new ideas.
8
SPEAKER_00
So for instance, next to my bed, I have a voice recorder.
8
SPEAKER_00
And it happens to me often, like I wake up during the night, and I have some idea.
8
SPEAKER_00
In the past, I was writing them down on my phone.
8
SPEAKER_00
But that means turning on the screen, and that wakes me up.
8
SPEAKER_00
Or like pulling a paper, which requires turning on the light.
8
SPEAKER_00
These days, I just start recording it.
8
SPEAKER_00
What do you think?
8
SPEAKER_00
I don t know if you know who Jim Keller is.
8
SPEAKER_00
I know Jim Keller.
8
SPEAKER_02
He s a big proponent of thinking hard on a problem right before sleep so that he can sleep through it and solve it in his sleep or like come up with radical stuff in his sleep.
8
SPEAKER_02
He was trying to get me to do this.
8
SPEAKER_00
So it happened from my experience perspective.
8
SPEAKER_00
It happened to me many times during the high school days when I was doing mathematics that I had the solution to my problem as I woke up.
8
SPEAKER_00
At the moment regarding thinking hard about the problem is.
8
SPEAKER_00
I m trying to actually devote substantial amount of time to think about important problems, not just before the sleep.
8
SPEAKER_00
I can organizing amount of the huge chunks of time such that I m not constantly working on the urgent problems, but I actually have time to think about the important one.
8
SPEAKER_00
So you do it naturally.
8
SPEAKER_02
But his idea is that you kind of.
8
SPEAKER_02
prime your brain to make sure that that s the focus.
8
SPEAKER_02
You know, oftentimes people have other worries in their life that s not fundamentally deep problems.
8
SPEAKER_02
Like, I don t know, just stupid drama in your life and even at work, all that kind of stuff.
8
SPEAKER_02
He wants to kind of pick the most important problem that you re thinking about and go to bed on that.
8
SPEAKER_00
I think that s wise.
8
SPEAKER_00
I mean, the other thing that comes to my mind is also I feel the most fresh in the morning.
8
SPEAKER_00
So during the morning, I try to work on the most important things rather than just being pulled by urgent things or checking email or so.
8
SPEAKER_02
What do you do with the because I ve been doing the voice recorder thing, too, but I end up recording so many messages.
8
SPEAKER_02
It s hard to organize.
8
SPEAKER_00
I have the same problem.
8
SPEAKER_00
Now I have heard that Google Pixel is really good in transcribing text, and I might get a Google Pixel just for the sake of transcribing text.
8
SPEAKER_02
Yeah, people listening to this, if you have a good voice recorder suggesting that transcribe, please let me know.
8
SPEAKER_02
Some of it is, this has to do with OpenAI codecs, too.
8
SPEAKER_02
Some of it is simply the friction.
8
SPEAKER_02
I need...
8
SPEAKER_02
apps that remove that friction between voice and the organization of the resulting transcripts and all that kind of stuff.
8
SPEAKER_02
But yes, you re right.
8
SPEAKER_02
Absolutely.
8
SPEAKER_02
Like during for me, it s walking, sleep too, but walking and running, especially running, get a lot of thoughts during running and there s no good mechanism for recording thoughts.
8
SPEAKER_00
So one more thing that I do, I have a separate phone.
8
SPEAKER_00
which has no apps.
8
SPEAKER_00
Maybe it says like Audible or let s say Kindle.
8
SPEAKER_00
No one has this phone number.
8
SPEAKER_00
This is kind of my meditation phone.
8
SPEAKER_00
And I try to expand the amount of time that that s the phone that I m having.
8
SPEAKER_00
It has also Google Maps if I need to go somewhere.
8
SPEAKER_00
And I also use this phone to write down ideas.
8
SPEAKER_02
Ah, that s a really good idea.
8
SPEAKER_02
That s a really good idea.
8
SPEAKER_00
Often actually what I end up doing is even sending a message from that phone to the other phone.
8
SPEAKER_00
So that s actually my way of recording messages or I just put them into notes.
8
SPEAKER_02
I love it.
8
SPEAKER_02
What advice would you give to a young person, high school, college, about how to be successful?
8
SPEAKER_02
You ve done a lot of incredible things in the past decade.
8
SPEAKER_02
So maybe you have some... Something.
8
SPEAKER_00
There might be something.
8
SPEAKER_02
There might be something.
8
SPEAKER_00
I mean, it might sound simplistic or so, but I would say literally just...
8
SPEAKER_00
Follow your passion, double down on it.
8
SPEAKER_00
And if you don t know what s your passion, just figure out what could be a passion.
8
SPEAKER_00
So the step might be an exploration.
8
SPEAKER_00
When I was in elementary school, it was math and chemistry.
8
SPEAKER_00
And I remember for some time, I gave up on math because my school teacher, she told me that I m dumb.
8
SPEAKER_00
And I guess maybe an advice would be just ignore people if they tell you that you re dumb.
8
SPEAKER_02
You mentioned something offline about chemistry and explosives.
8
SPEAKER_02
What was that about?
8
SPEAKER_00
So let s see.
8
SPEAKER_00
So the story goes like that.
8
SPEAKER_00
I got into chemistry, maybe I was like a second grade of my elementary school, third grade.
8
SPEAKER_00
I started going to chemistry classes.
8
SPEAKER_00
I really love building stuff.
8
SPEAKER_00
And I did all the experiments that they describe in the book, like, you know, how to create oxygen
8
SPEAKER_00
with vinegar and baking soda or so.
8
SPEAKER_00
So I did all the experiments.
8
SPEAKER_00
And at some point I was, you know, so what s next?
8
SPEAKER_00
What can I do?
8
SPEAKER_00
And explosives, they also, it s like a, you have a clear reward signal, you know, if the thing worked or not.
8
SPEAKER_00
So I remember at first I got interested in producing hydrogen.
8
SPEAKER_00
That was kind of funny experiment from school.
8
SPEAKER_00
You can just burn it.
8
SPEAKER_00
And then I moved to nitroglycerin.
8
SPEAKER_00
So that s also relatively easy to synthesize.
8
SPEAKER_00
I started producing essentially dynamite and detonating it with a friend.
8
SPEAKER_00
I remember there was at first like maybe two attempts that I went with a friend to detonate what we built.
8
SPEAKER_00
And it didn t work out.
8
SPEAKER_00
And like a third time he was like, ah, it won t work.
8
SPEAKER_00
Like, let s don t waste time.
8
SPEAKER_00
And now we were.
8
SPEAKER_00
I was carrying this, you know, that tube with dynamite, I don t know, pound or so, dynamite in my backpack.
8
SPEAKER_00
We re like riding on the bike to the edges of the city.
8
SPEAKER_00
Attempt number three.
8
SPEAKER_00
This would be attempt number three.
8
SPEAKER_00
Attempt number three.
8
SPEAKER_00
Now we dig a hole to put it inside.
8
SPEAKER_00
It actually had the, you know, electrical detonator.
8
SPEAKER_00
We draw a cable behind the tree.
8
SPEAKER_00
I even, I never, I haven t ever seen like a explosion before.
8
SPEAKER_00
So I thought that there will be a lot of sound, but you know, we re like laying down and I m holding the cable and the battery at some point, you know, we kind of like a three to one and I just connected it and it felt like at the ground shake, it was like a more like a sound.
8
SPEAKER_00
And then the soil started kind of lifting up and started falling on us.
8
SPEAKER_00
Yeah.
8
SPEAKER_00
Wow.
8
SPEAKER_00
And then the friend said, let s make sure next time we have helmets.
8
SPEAKER_00
But also, I m happy that nothing happened to me.
8
SPEAKER_00
It could have been the case that I lost the limbo or so.
8
SPEAKER_02
Yeah.
8
SPEAKER_02
But that s childhood of an engineering mind with a strong reward signal.
8
SPEAKER_02
of an explosion.
8
SPEAKER_02
I love it.
8
SPEAKER_02
There s some aspect of chemists, the chemists I know, like my dad, with plasma chemistry, plasma physics, he was very much into explosives, too.
8
SPEAKER_02
It s a worrying quality of people that work in chemistry that they love.
8
SPEAKER_02
I think it is that, exactly, is the strong signal that the thing worked.
8
SPEAKER_02
There is no doubt.
8
SPEAKER_02
There s no doubt.
8
SPEAKER_02
There s some magic.
8
SPEAKER_02
It s almost like a reminder that physics works, that chemistry works.
8
SPEAKER_02
It s cool.
8
SPEAKER_02
It s almost like a little glimpse at nature that you yourself engineer.
8
SPEAKER_02
That s why I really like artificial intelligence, especially robotics, is you create a little piece of nature.
8
SPEAKER_00
And in some sense, even for me with explosives, the motivation was creation rather than destruction.
8
SPEAKER_00
Yes, exactly.
8
SPEAKER_02
In terms of advice, I forgot to ask about just machine learning and deep learning for people who are specifically interested in machine learning.
8
SPEAKER_02
How would you recommend they get into the field?
8
SPEAKER_00
So I would say reimplement everything.
8
SPEAKER_00
And also there is plenty of courses.
8
SPEAKER_00
So like from scratch?
8
SPEAKER_00
So on different levels of abstraction in some sense, but I would say reimplement something from scratch, reimplement something from a paper, reimplement something, you know, from podcasts that you have heard about.
8
SPEAKER_00
I would say that s a powerful way to understand things.
8
SPEAKER_00
So it s often the case that you read the description and you think you understand.
8
SPEAKER_00
but you truly understand once you build it, then you actually know what really matters in the description.
8
SPEAKER_02
Is there particular topics that you find people just fall in love with?
8
SPEAKER_02
I ve seen
8
SPEAKER_02
I tend to really enjoy reinforcement learning because it s, it s much more, it s much easier to get to a point where you feel like you created something special, like, like fun games, kind of things that are rewarding.
8
SPEAKER_02
It s rewarding.
8
SPEAKER_02
Yeah.
8
SPEAKER_02
As opposed to like re-implementing from scratch, more like supervised learning kind of things.
8
SPEAKER_02
It s, it s yeah.
8
SPEAKER_00
So, you know,
8
SPEAKER_00
if someone would optimize for things to be rewarding, then it feels that the things that are somewhat generative, they have such a property.
8
SPEAKER_00
So you have, for instance, adversarial networks, or you have just even generative language models.
8
SPEAKER_00
And you can even see
8
SPEAKER_00
Internally, we have seen this thing with our releases.
8
SPEAKER_00
So we released recently two models.
8
SPEAKER_00
There is one model called DALI that generates images.
8
SPEAKER_00
And there is other model called CLIP that actually you provide various possibilities what could be the answer to what is on the picture.
8
SPEAKER_00
And it can tell you which one is the most likely.
8
SPEAKER_00
And in some sense, in case of the first one, Dali, it is very easy for you to understand that actually there is magic going on.
8
SPEAKER_00
And in the in case of the second one, even though it is insanely powerful and, you know, people from vision community, they as they started probing it inside, they actually understood
8
SPEAKER_00
how far it goes, it s difficult for person at first to see how well it works.
8
SPEAKER_00
And that s the same, as you said, that in case of supervised learning models, you might not kind of see, or it s not that easy for you to understand the strength.
8
SPEAKER_02
Even though you don t believe in magic, to see the magic.
8
SPEAKER_00
To see the magic, yeah.
8
SPEAKER_02
The generative, that s really brilliant.
8
SPEAKER_02
So anything that s generative, because then you are at the core of the creation.
8
SPEAKER_02
You get to experience creation without much effort, unless you have to do it from scratch.
8
SPEAKER_00
And it feels that humans are wired.
8
SPEAKER_00
There is some level of reward for creating stuff.
8
SPEAKER_00
Of course, different people have a different weight on this reward.
8
SPEAKER_02
In the big objective function.
8
SPEAKER_00
In the big objective function, of a person.
8
SPEAKER_02
Of a person.
8
SPEAKER_02
You wrote that beautiful is what you intensely pay attention to.
8
SPEAKER_02
Even a cockroach is beautiful if you look very closely.
8
SPEAKER_02
Can you expand on this?
8
SPEAKER_02
What is...
8
SPEAKER_00
beauty so What I m I wrote here actually corresponds to my subjective experience that I had through extended periods of meditation It s it s pretty crazy that at some point the meditation gets you to the place that you have really increased focus increased attention and then you look at the very simple objects that were all the time around you can look at the table or on the pen and
8
SPEAKER_00
or at the nature.
8
SPEAKER_00
And you notice more and more details, and it becomes very pleasant to look at it.
8
SPEAKER_00
And it once again, it kind of reminds me of my childhood.
8
SPEAKER_00
Like a just pure joy of being.
8
SPEAKER_00
It s also, I have seen even the reverse effect that by default, regardless of what we possess, we very quickly get used to it.
8
SPEAKER_00
And you can have a very beautiful house.
8
SPEAKER_00
And if you don t put sufficient effort, you re just going to get used to it.
8
SPEAKER_00
And it doesn t bring any more joy, regardless of what you have.
8
SPEAKER_02
Yeah.
8
SPEAKER_02
Well, actually, I find that material possessions get in the way of that experience of pure joy.
8
SPEAKER_02
So I ve always been very fortunate.
8
SPEAKER_02
to just find joy in simple things, just like you re saying.
8
SPEAKER_02
Just like, I don t know, objects in my life, just stupid objects, like this cup, like thing, you know, just objects.
8
SPEAKER_02
Sounds, okay, I m not being eloquent, but literally objects in the world.
8
SPEAKER_02
They re just full of joy because it s like, I can t believe, one, I can t believe that I m fortunate enough to be alive to experience these objects.
8
SPEAKER_02
And then two, I can t believe humans are clever enough to have built these objects.
8
SPEAKER_02
the hierarchy of pleasure that that provides is infinite.
8
SPEAKER_00
I mean, even if you look at the cup of water, so, you know, you see first like a level of like a reflection of light, but then you think, you know, man, there s like a trillions upon of trillions of particles bouncing against each other.
8
SPEAKER_00
There is also the tension on the surface that, you know, if the back could like stand on it and move around,
8
SPEAKER_00
And you think it also has this like a magical property that as you decrease temperature, it actually expands in volume, which allows for the lakes to freeze on the surface and at the bottom to have actually not freeze, which allows for life like that.
8
SPEAKER_00
crazy.
8
SPEAKER_00
Yeah.
8
SPEAKER_00
You look in detail at some object and you think actually, you know, this table that was just the figment of someone s imagination at some point.
8
SPEAKER_00
And then there was like thousands of people involved to actually manufacture it and put it here.
8
SPEAKER_00
And by default, no one cares.
8
SPEAKER_02
And then you can start thinking about evolution, how it all started from single cell organisms that led to this table.
8
SPEAKER_00
And these thoughts, they give me life appreciation.
8
SPEAKER_00
Yeah, exactly.
8
SPEAKER_00
And even lack of thoughts, just the pure raw signal also gives the life appreciation.
8
SPEAKER_02
See, the thing is, and then that s coupled for me with the sadness that the whole ride ends.
8
SPEAKER_02
and perhaps is deeply coupled in that the fact that this experience, this moment ends, gives it an intensity that I m not sure I would otherwise have.
8
SPEAKER_02
So in that same way, I try to meditate on my own death often.
8
SPEAKER_02
Do you think about your mortality?
8
SPEAKER_02
Are you afraid of death?
8
SPEAKER_00
So fear of death is like one of the most fundamental fears that each of us has.
8
SPEAKER_00
We might be not even aware of it.
8
SPEAKER_00
It requires to look inside to even recognize that it s out there.
8
SPEAKER_00
And there is still, let s say, this property of nature that if things would last forever, then they would be also boring to us.
8
SPEAKER_00
The fact that the things change in some way gives any meaning to them.
8
SPEAKER_00
I also found out that it seems to be very healing to people to have
8
SPEAKER_00
these short experiences, like, I guess, psychedelic experiences in which they experience death of self, in which they let go of this fear, and then maybe can even increase their appreciation of the moment.
8
SPEAKER_00
It seems that many people, they
8
SPEAKER_00
They can easily comprehend the fact that the money is finite while they don t see that time is finite.
8
SPEAKER_00
I have this like a discussion with Ilya from time to time.
8
SPEAKER_00
He s like, you know, man like a
8
SPEAKER_00
the life will pass very fast.
8
SPEAKER_00
At some point I will be 40, 50, 60, 70 and then it s over.
8
SPEAKER_00
This is true, which also makes me believe that, you know, that every single moment it is so unique that should be appreciated.
8
SPEAKER_00
And this also makes me think that I should be acting on my life because otherwise it will pass.
8
SPEAKER_00
I also like this framework of thinking from Jeff Bezos on regret minimization.
8
SPEAKER_00
Like I would like if I will be at that deathbed to look back on my life and not regret that I haven t done something.
8
SPEAKER_00
It s usually you might regret that you haven t tried.
8
SPEAKER_00
I m fine with failing.
8
SPEAKER_02
I haven t tried.
8
SPEAKER_02
Well, it s the Nietzsche eternal recurrence.
8
SPEAKER_02
He tried to live a life that if you had to live it infinitely many times, you d be okay with that kind of life.
8
SPEAKER_02
So try to live it optimally.
9
SPEAKER_01
I can say that it s almost like I m unbelievable to me where I am in my life.
9
SPEAKER_01
I m extremely grateful for actually people whom I met.
9
SPEAKER_01
I would say I think that I m decently smart and so on.
9
SPEAKER_01
But I think that actually to great extent where I am has to do with the people who I met.
9
SPEAKER_01
Would you be okay if after this conversation you died?
9
SPEAKER_01
So if I m dead, then it kind of, I don t have a choice anymore.
9
SPEAKER_01
So in some sense, there s like a plenty of things that I would like to try out in my life.
9
SPEAKER_01
I feel that, you know, I m gradually going one by one and I m just doing them.
9
SPEAKER_01
I think that the list will be always infinite.
9
SPEAKER_00
Yeah.
9
SPEAKER_00
So might as well go today.
9
SPEAKER_01
Yeah, I mean, to be clear, I m not looking forward to die.
9
SPEAKER_01
I would say if there is no choice, I would accept it.
9
SPEAKER_01
But like, in some sense, if there would be a choice, if there would be possibility to live, I would fight for living.
9
SPEAKER_00
I find it s more honest than real to think about, you know, dying today at the end of the day.
9
SPEAKER_00
that seems to me, at least to my brain, more on a slap in the face, as opposed to, I still have 10 years, like today, then I m much more about appreciating the cup and the table and so on, and less about like silly worldly accomplishments and all those kinds of things.
9
SPEAKER_01
we have in the company a person who say at some point found out that they have cancer.
9
SPEAKER_01
And that also gives, you know, huge perspective with respect to what matters now.
9
SPEAKER_01
Yeah.
9
SPEAKER_01
And, you know, often people in situations like that, they conclude that actually what matters is human connection and love.
9
SPEAKER_00
And that s people conclude also if you have kids, kids, family.
9
SPEAKER_00
You, I think, tweeted, we don t assign the minus infinity reward to our death.
9
SPEAKER_00
Such a reward would prevent us from taking any risk.
9
SPEAKER_00
We wouldn t be able to cross a road in fear of being hit by a car.
9
SPEAKER_00
So in the objective function, you mentioned fear of death might be fundamental to the human condition.
9
SPEAKER_01
So as I said, let s assume that there are reward functions in our brain.
9
SPEAKER_01
And the interesting thing is even realization how different reward functions can play with your behavior.
9
SPEAKER_01
As a matter of fact, I wouldn t say that you should assign infinite negative reward to anything because that messes up the math.
9
SPEAKER_01
The math doesn t work out.
9
SPEAKER_01
It doesn t work out.
9
SPEAKER_01
And it s, as you said, even, you know, government or some insurance companies, you said they assign nine billion, nine million dollars to human life.
9
SPEAKER_01
Yeah.
9
SPEAKER_01
And I m just saying it with respect to.
9
SPEAKER_01
That might be a hard statement to ourselves, but in some sense that there is a finite value of our own life.
9
SPEAKER_01
I m trying to put it from perspective of being less of being more egoless and realizing fragility of my own life.
9
SPEAKER_01
And in some sense, the fear of death might prevent you from acting because anything can cause death.
9
SPEAKER_00
Yeah, and I m sure, actually, if you were to put death in the objective function, there s probably so many aspects to death and fear of death and realization of death and mortality.
9
SPEAKER_00
There s just whole components of finiteness of not just your life, but every experience and so on that you re going to have to formalize mathematically.
9
SPEAKER_01
And also, you know, that might lead to
9
SPEAKER_01
You re spending a lot of compute cycles on this, like, deliberating this terrible future instead of experiencing now.
9
SPEAKER_01
And then in some sense is also kind of unpleasant simulation to run in your head.
9
SPEAKER_00
Yeah.
9
SPEAKER_00
Do you think there s an objective function that describes the entirety of human life?
9
SPEAKER_00
So, you know, usually the way you ask that is what is the meaning of life?
9
SPEAKER_00
Is there a universal objective functions that captures the why of life?
9
SPEAKER_01
So yeah, I mean, I suspect that they will ask this question.
9
SPEAKER_01
But it s also a question that I ask myself many, many times.
9
SPEAKER_01
See, I can tell you a framework that I have these days to think about this question.
9
SPEAKER_01
So I think that fundamentally, meaning of life has to do with some of our reward functions that we have in brain.
9
SPEAKER_01
And they might have to do with, let s say, for instance, curiosity or human connection, which might mean understanding others.
9
SPEAKER_01
It s also possible for a person to slightly modify their reward function.
9
SPEAKER_01
Usually, they mostly stay fixed, but it s possible to modify a reward function.
9
SPEAKER_01
And you can pretty much choose.
9
SPEAKER_01
So in some sense, reward functions, optimizing reward functions, they will give you life satisfaction.
9
SPEAKER_01
Is there some randomness in the function?
9
SPEAKER_01
I think when you are born, there is some randomness.
9
SPEAKER_01
Like, you can see that some people, for instance, they care more about building stuff.
9
SPEAKER_01
Some people care more about caring for others.
9
SPEAKER_01
Some people, there are all sorts of default reward functions.
9
SPEAKER_01
And then in some sense, you can ask yourself, what is the satisfying way for you to go after this reward function?
9
SPEAKER_01
And you just go after this reward function.
9
SPEAKER_01
And some people also ask, are these reward functions real?
9
SPEAKER_01
I almost think about it as
9
SPEAKER_01
Let s say, if you would have to discover mathematics, in mathematics, you are likely to run into various objects, like complex numbers or differentiation, some other objects.
9
SPEAKER_01
And these are very natural objects that arise.
9
SPEAKER_01
And similarly, the reward functions that we are having in our brain, they are somewhat very natural, that there is a reward function for understanding
9
SPEAKER_01
like a comprehension, curiosity and so on.
9
SPEAKER_01
So in some sense they are in the same way natural as they re natural objects in mathematics.
9
SPEAKER_00
Interesting.
9
SPEAKER_00
So you know there s the old sort of debate, is mathematics invented or discovered?
9
SPEAKER_00
You re saying reward functions are discovered.
9
SPEAKER_00
So nature...
9
SPEAKER_01
So if nature provided some, you can still, let s say, expand it throughout the life.
9
SPEAKER_01
Some of the reward functions, they might be futile.
9
SPEAKER_01
Like, for instance, there might be a reward function, maximize amount of wealth.
9
SPEAKER_00
Yeah.
9
SPEAKER_01
And this is more like a learned reward function.
9
SPEAKER_01
But we know also that some reward functions, if you optimize them, you won t be quite satisfied.
9
SPEAKER_00
Well, I don t know which part of your reward function resulted in you coming today, but I am deeply appreciative that you did spend your valuable time with me.
9
SPEAKER_00
Wojciech, it s really fun talking to you.
9
SPEAKER_00
You re brilliant, you re a good human being, and it s an honor to meet you and an honor to talk to you.
9
SPEAKER_00
Thanks for talking today, brother.
9
SPEAKER_01
Thank you, Lex, a lot.
9
SPEAKER_01
I appreciated your questions, curiosity.
9
SPEAKER_01
I had a lot of time being here.
9
SPEAKER_00
Thanks for listening to this conversation with Wojciech Saremba.
9
SPEAKER_00
To support this podcast, please check out our sponsors in the description.
9
SPEAKER_00
And now, let me leave you with some words from Arthur C. Clarke, who is the author of 2001, A Space Odyssey.
9
SPEAKER_00
It may be that our role on this planet is not to worship God, but to create him.
9
SPEAKER_00
Thank you for listening, and I hope to see you next time.