0
SPEAKER_01
The following is a conversation with Andrej Karpathy, previously the director of AI at Tesla, and before that at OpenAI and Stanford.
0
SPEAKER_01
He is one of the greatest scientists, engineers, and educators in the history of artificial intelligence.
0
SPEAKER_01
And now, a quick few second mention of each sponsor.
0
SPEAKER_01
Check them out in the description.
0
SPEAKER_01
It s the best way to support this podcast.
0
SPEAKER_01
We got 8sleep for naps, BetterHelp for mental health, Fundrise for real estate investing, and Athletic Greens for nutrition.
0
SPEAKER_01
Choose wisely, my friends.
0
SPEAKER_01
And now, onto the full ad reads.
0
SPEAKER_01
As always, no ads in the middle.
0
SPEAKER_01
I try to make this interesting, but if you skip them, please still check out our sponsors.
0
SPEAKER_01
I enjoy their stuff.
0
SPEAKER_01
Maybe you will, too.
0
SPEAKER_01
This episode is sponsored by Eight Sleep and its new Pod 3 mattress.
0
SPEAKER_01
I am recording this in a hotel.
0
SPEAKER_01
In fact, given some complexities of my life, this is the middle of the night, 4 a.m.
0
SPEAKER_01
I m sitting in an empty hotel room, yelling at a microphone.
0
SPEAKER_01
This, my friends, is my life.
0
SPEAKER_01
I do usually feel good about myself at 4am, but not with two cups of coffee in me.
0
SPEAKER_01
And the reason I feel good is because I m going to go to sleep soon, and I ve accomplished a lot.
0
SPEAKER_01
This is true today, except for the sleep soon part because I think I m going to an airport at some point soon.
0
SPEAKER_01
It doesn t matter.
0
SPEAKER_01
What matters is I m not even going to sleep here and that s great because in a hotel I don t have an 8 sleep bed that can cool itself.
0
SPEAKER_01
At home I do and that s where I m headed.
0
SPEAKER_01
I m headed home.
0
SPEAKER_01
Anyway, check it out and get special savings when you go to 8sleep.com slash Lex.
0
SPEAKER_01
This episode is also brought to you by BetterHelp, spelled H-E-L-P, help.
0
SPEAKER_01
I m a huge fan of talk therapy.
0
SPEAKER_01
I think of podcasting as a kind of talk therapy.
0
SPEAKER_01
So I m a huge fan of listening to podcasts.
0
SPEAKER_01
In fact, that s how I think of doing a podcast myself.
0
SPEAKER_01
I just get to have front row seats to a thing I love.
0
SPEAKER_01
And it s actually just the process of talking that reveals something about the mind.
0
SPEAKER_01
I think that s what good talk therapy is, is it guided by a professional therapist, it helps you reveal to yourself something about your mind.
0
SPEAKER_01
Just lay it all out on the table.
0
SPEAKER_01
So yeah, you should definitely use the best method of talk therapy, the best meaning the most accessible, at least to try it.
0
SPEAKER_01
If not, to make it a regular part of your life, that s what BetterHelp does.
0
SPEAKER_01
Check them out at betterhelp.com slash Lex and save on your first month.
0
SPEAKER_01
This episode is also brought to you by Fundrise, spelled F-U-N-D-R-I-S-E.
0
SPEAKER_01
It s a platform that allows you to invest in private real estate.
0
SPEAKER_01
We live in hard times, folks, for many different reasons, but one of them is financial.
0
SPEAKER_01
And one way to protect yourself in difficult times is diversify your investments.
0
SPEAKER_01
Private real estate is one of the things I believe you should diversify into.
0
SPEAKER_01
And when you do, you should use tools that look like they re made in the 21st century, which a lot of investment, even like online investment websites and services seem to be designed by the same people that designed the original ATMs.
0
SPEAKER_01
That s not the case with Fundrise.
0
SPEAKER_01
Super easy to use, accessible, over 150,000 investors use it.
0
SPEAKER_01
Their team vets and manages all their real estate projects.
0
SPEAKER_01
You can track your portfolio s performance on their website and see updates as properties across the country are acquired, improved, and operated.
0
SPEAKER_01
Anyway, check out Fundrise.
0
SPEAKER_01
It takes just a few minutes to get started at fundrise.com slash lex.
0
SPEAKER_01
This show is brought to you by Athletic Greens and its AG1 drink, which is an all-in-one daily drink to support better health and peak performance.
0
SPEAKER_01
I have to be honest, I completely forgot to bring Athletic Greens with me as I m traveling now, and I miss it.
0
SPEAKER_01
It s not just good for my nutritional base and needs, it s good for my soul.
0
SPEAKER_01
It s part of the daily habit of life.
0
SPEAKER_01
And when you don t have that habit, the routine stuff is off.
0
SPEAKER_01
So it s good to just put that into your daily routine to make sure that you re getting the vitamins, the nutrition that you need, no matter the dietary.
0
SPEAKER_01
the workload, the athletic endeavors that you partake in.
0
SPEAKER_01
I don t know, it s kind of incredible.
0
SPEAKER_01
And yeah, that s what Athletic Greens is for me.
0
SPEAKER_01
They ll give you a one month supply of fish oil when you sign up at athleticgreens.com slash Lex.
0
SPEAKER_01
This is the Lex Friedman Podcast.
0
SPEAKER_01
To support it, please check out our sponsors.
0
SPEAKER_01
And now, dear friends, here s Andrej Karpathy.
0
SPEAKER_01
What is a neural network?
0
SPEAKER_01
And why does it seem to do such a surprisingly good job of learning?
0
SPEAKER_00
What is a neural network?
0
SPEAKER_00
It s a mathematical abstraction of
0
SPEAKER_00
the brain, I would say.
0
SPEAKER_00
That s how it was originally developed.
0
SPEAKER_00
At the end of the day, it s a mathematical expression, and it s a fairly simple mathematical expression when you get down to it.
0
SPEAKER_00
It s basically a sequence of matrix multiplies, which are really dot products mathematically, and some nonlinearities thrown in.
0
SPEAKER_00
And so it s a very simple mathematical expression, and it s got knobs in it.
0
SPEAKER_00
many knobs many knobs and these knobs are loosely related to basically the synapses in your brain they re trainable they re modifiable and so the idea is like we need to find the setting of the knobs that makes the neural net uh do whatever you want it to do like classify images and so on and so there s not too much mystery i would say in it like um you might think that
0
SPEAKER_00
basically don t want to endow it with too much meaning with respect to the brain and how it works.
0
SPEAKER_00
It s really just a complicated mathematical expression with knobs, and those knobs need a proper setting for it to do something desirable.
0
SPEAKER_01
Yeah, but poetry is just a collection of letters with spaces, but it can make us feel a certain way.
0
SPEAKER_01
And in that same way, when you get a large number of knobs together, whether it s inside the brain or inside a computer, they seem to surprise us with their power.
0
SPEAKER_00
Yeah, I think that s fair.
0
SPEAKER_00
So basically, I m underselling it by a lot because you definitely do get very surprising emergent behaviors out of these neural nets when they re large enough and trained on complicated enough problems, like say, for example, the next word prediction in a massive data set from the internet.
0
SPEAKER_00
And then these neural nets take on pretty surprising magical properties.
0
SPEAKER_00
Yeah, I think it s kind of interesting how much you can get out of even very simple mathematical formalism.
0
SPEAKER_01
When your brain right now is talking, is it doing next word prediction?
0
SPEAKER_01
Or is it doing something more interesting?
0
SPEAKER_00
Well, it s definitely some kind of a generative model that s GPT-like and prompted by you.
0
SPEAKER_00
So you re giving me a prompt, and I m kind of responding to it in a generative way.
0
SPEAKER_01
And by yourself, perhaps, a little bit?
0
SPEAKER_01
Are you adding extra prompts from your own memory inside your head?
0
SPEAKER_00
Well, it definitely feels like you re referencing some kind of a declarative structure of memory and so on.
0
SPEAKER_00
you re putting that together with your prompt and giving away some answers.
0
SPEAKER_01
How much of what you just said has been said by you before?
0
SPEAKER_00
Nothing, basically, right?
0
SPEAKER_01
No, but if you actually look at all the words you ve ever said in your life, and you do a search, you ll probably have said a lot of the same words in the same order before.
0
SPEAKER_00
Yeah, could be.
0
SPEAKER_00
I mean, I m using phrases that are common, etc., but I m remixing it into a pretty sort of unique sentence at the end of the day.
0
SPEAKER_00
But you re right, definitely there s like a ton of remixing.
0
SPEAKER_01
It s like Magnus Carlsen said, I m rated 2,900 whatever, which is pretty decent.
0
SPEAKER_01
I think you re talking very, you re not giving enough credit to your own nuts here.
0
SPEAKER_01
Why do they seem to, what s your best intuition about this emergent behavior?
0
SPEAKER_00
I mean, it s kind of interesting because I m simultaneously underselling them, but I also feel like there s an element to which I m over... It s actually kind of incredible that you can get so much emergent magical behavior out of them despite them being so simple mathematically.
0
SPEAKER_00
So I think those are kind of like two surprising statements that are kind of juxtaposed together.
0
SPEAKER_00
And I think basically what it is, is we are actually fairly good at optimizing these neural nets.
0
SPEAKER_00
And when you give them a hard enough problem, they are forced to learn very interesting solutions in the optimization.
0
SPEAKER_00
And those solutions basically have these emergent properties that are very interesting.
0
SPEAKER_01
There s wisdom and knowledge.
0
SPEAKER_01
in the knobs.
0
SPEAKER_01
And so this representation that s in the knobs, does it make sense to you intuitively that a large number of knobs can hold a representation that captures some deep wisdom about the data it has looked at?
0
SPEAKER_01
It s a lot of knobs.
0
SPEAKER_00
It s a lot of knobs.
0
SPEAKER_00
And somehow, you know, so speaking concretely, one of the neural nets that people are very excited about right now are GPTs, which are basically just next word prediction networks.
0
SPEAKER_00
So you consume a sequence of words from the internet and you try to predict the next word.
0
SPEAKER_00
And once you train these on a large enough data set,
0
SPEAKER_00
you can basically prompt these neural nets in arbitrary ways, and you can ask them to solve problems, and they will.
0
SPEAKER_00
So you can just tell them, you can make it look like you re trying to solve some kind of a mathematical problem, and they will continue what they think is the solution based on what they ve seen on the internet.
0
SPEAKER_00
And very often, those solutions look very remarkably consistent, look correct potentially.
0
SPEAKER_01
Do you still think about the brain side of it?
0
SPEAKER_01
So as neural nets is an abstraction, a mathematical abstraction of the brain, do you still draw wisdom from the biological neural networks?
0
SPEAKER_01
Or even the bigger question, so you re a big fan of biology and biological computation.
0
SPEAKER_01
What impressive thing is biology doing to you that computers are not yet, that gap?
0
SPEAKER_00
I would say I m definitely on, I m much more hesitant with the analogies to the brain than I think you would see potentially in the field.
0
SPEAKER_00
And I kind of feel like certainly the way neural networks started is everything stemmed from inspiration by the brain.
0
SPEAKER_00
But at the end of the day, the artifacts that you get after training, they are arrived at by a very different optimization process than the optimization process that gave rise to the brain.
0
SPEAKER_00
And so I think,
0
SPEAKER_00
I kind of think of it as a very complicated alien artifact.
0
SPEAKER_00
It s something different.
0
SPEAKER_00
I m sorry, the neural nets that we re training.
0
SPEAKER_00
They are a complicated alien artifact.
0
SPEAKER_00
I do not make analogies to the brain because I think the optimization process that gave rise to it is very different from the brain.
0
SPEAKER_00
So there was no multi-agent self-play kind of setup and evolution.
0
SPEAKER_00
It was an optimization that is basically what amounts to a compression objective on a massive amount of data.
0
SPEAKER_01
Okay, so artificial neural networks are doing compression, and biological neural networks are not really doing anything.
0
SPEAKER_01
They re an agent in a multi-agent self-play system that s been running for a very, very long time.
0
SPEAKER_00
That said, evolution has found that it is very useful to predict and have a predictive model in the brain.
0
SPEAKER_00
And so I think our brain utilizes something that looks like that as a part of it, but it has a lot more gadgets and gizmos and value functions and ancient nuclei that are all trying to make it survive and reproduce and everything else.
0
SPEAKER_01
And the whole thing through embryogenesis is built from a single cell.
0
SPEAKER_01
I mean, it s just, the code is inside the DNA, and it just builds it up, like the entire organism, with arms and a head and legs.
0
SPEAKER_01
And like it does it pretty well.
0
SPEAKER_01
It should not be possible.
0
SPEAKER_01
So there s some learning going on.
0
SPEAKER_01
There s some kind of computation going through that building process.
0
SPEAKER_01
I mean, I don t know where,
0
SPEAKER_01
if you were just to look at the entirety of history of life on Earth, where do you think is the most interesting invention?
0
SPEAKER_01
Is it the origin of life itself?
0
SPEAKER_01
Is it just jumping to eukaryotes?
0
SPEAKER_01
Is it mammals?
0
SPEAKER_01
Is it humans themselves, homo sapiens?
0
SPEAKER_01
The origin of intelligence or highly complex intelligence?
0
SPEAKER_01
Or is it all just a continuation of the same kind of process?
0
SPEAKER_00
Certainly, I would say it s an extremely remarkable story that I m only briefly learning about recently.
0
SPEAKER_00
Actually, you almost have to start at the formation of Earth and all of its conditions and the entire solar system and how everything is arranged with Jupiter and Moon and the habitable zone and everything.
0
SPEAKER_00
Then you have an active Earth that s turning over material.
0
SPEAKER_00
Then you start with abiogenesis and everything.
0
SPEAKER_00
It s all a pretty remarkable story.
0
SPEAKER_00
I m not sure that
0
SPEAKER_00
I can pick a single unique piece of it that I find most interesting.
0
SPEAKER_00
I guess for me as an artificial intelligence researcher, it s probably the last piece.
0
SPEAKER_00
We have lots of animals that are not building technological society, but we do.
0
SPEAKER_00
It seems to have happened very quickly.
0
SPEAKER_00
It seems to have happened very recently.
0
SPEAKER_00
something very interesting happened there that I don t fully understand.
0
SPEAKER_00
I almost understand everything else, I think intuitively, but I don t understand exactly that part and how quick it was.
0
SPEAKER_01
Both explanations would be interesting.
0
SPEAKER_01
One is that this is just a continuation of the same kind of process.
0
SPEAKER_01
There s nothing special about humans.
0
SPEAKER_01
That would be, deeply understanding that would be very interesting, that we think of ourselves as special, but it was obvious.
0
SPEAKER_01
It was already written in the code that you would have
0
SPEAKER_01
greater and greater intelligence emerging.
0
SPEAKER_01
And then the other explanation, which is something truly special happened, something like a rare event, whether it s like crazy rare event, like a space odyssey, what would it be?
0
SPEAKER_01
See, if you say like the invention of fire,
0
SPEAKER_01
or the, as Richard Rangham says, the beta males deciding a clever way to kill the alpha males by collaborating, so just optimizing the collaboration, the multi-agent aspect of the multi-agent.
0
SPEAKER_01
and that really being constrained on resources and trying to survive, the collaboration aspect is what created the complex intelligence.
0
SPEAKER_01
But it seems like it s a natural outgrowth of the evolution process.
0
SPEAKER_01
What could possibly be a magical thing that happened, like a rare thing that would say that humans are actually, human-level intelligence is actually a really rare thing in the universe?
0
SPEAKER_00
Yeah, I m hesitant to say that it is rare, by the way, but it definitely seems like
0
SPEAKER_00
kind of like a punctuated equilibrium where you have lots of exploration and then you have certain leaps, sparse leaps in between.
0
SPEAKER_00
Of course, origin of life would be one, DNA, sex, eukaryotic system, eukaryotic life, the endosymbiosis event where the Archeon ate little bacteria, just the whole thing, and then of course emergence of consciousness and so on.
0
SPEAKER_00
It seems like definitely there are sparse events where massive amount of progress was made, but yeah, it s kind of hard to pick one
0
SPEAKER_01
So you don t think humans are unique?
0
SPEAKER_01
Gotta ask you, how many intelligent alien civilizations do you think are out there?
0
SPEAKER_01
And is their intelligence different or similar to ours?
0
SPEAKER_00
Yeah.
0
SPEAKER_00
I ve been preoccupied with this question quite a bit recently, basically the Fermi paradox and just thinking through.
0
SPEAKER_00
And the reason actually that I am very interested in the origin of life is fundamentally trying to understand how common it is that there are technological societies out there in space.
0
SPEAKER_00
And the more I study it, the more I think that
0
SPEAKER_00
there should be quite a lot.
0
SPEAKER_01
Why haven t we heard from them?
0
SPEAKER_01
Because I agree with you.
0
SPEAKER_01
It feels like I just don t see why what we did here on Earth is so difficult to do.
0
SPEAKER_00
Yeah, and especially when you get into the details of it.
0
SPEAKER_00
I used to think Origin of Life was very
0
SPEAKER_00
It was this magical rare event, but then you read books like, for example, Nick Lane, The Vital Question, Life Ascending, etc.
0
SPEAKER_00
and he really gets in and he really makes you believe that this is not that rare.
1
SPEAKER_00
basic chemistry.
1
SPEAKER_01
You have an active earth and you have your alkaline vents and you have lots of alkaline waters mixing with acidic ocean and you have your proton gradients and you have little porous pockets of these alkaline vents that concentrate chemistry and basically as you step through all of these little pieces you start to understand that actually this is not that
1
SPEAKER_01
crazy.
1
SPEAKER_01
You could see this happen on other systems.
1
SPEAKER_01
And he really takes you from just a geology to primitive life, and he makes it feel like it s actually pretty plausible.
1
SPEAKER_01
And also, the origin of life was actually fairly fast after formation of Earth.
1
SPEAKER_01
If I remember correctly, just a few hundred million years or something like that, after basically when it was possible, life actually arose.
1
SPEAKER_01
And so that makes me feel like that is not the constraint, that is not the limiting variable, and that life should actually be fairly common.
1
SPEAKER_01
And then, you know, where the drop-offs are is very interesting to think about.
1
SPEAKER_01
I currently think that there s no major drop-offs, basically, and so there should be quite a lot of life.
1
SPEAKER_01
And basically, where that brings me to, then, is the only way to reconcile the fact that we haven t found anyone and so on is that
1
SPEAKER_01
We just can t see them.
1
SPEAKER_01
We can t observe them.
1
SPEAKER_00
Just a quick, brief comment.
1
SPEAKER_00
Nick Lane and a lot of biologists I talk to, they really seem to think that the jump from bacteria to more complex organisms is the hardest jump.
1
SPEAKER_01
The eukaryotic life, basically.
1
SPEAKER_00
Yeah.
1
SPEAKER_00
which I get it, they re much more knowledgeable than me about the intricacies of biology, but that seems like crazy, because how many single cell organisms are there, and how much time you have, surely it s not that difficult.
1
SPEAKER_00
In a billion years, it s not even that long.
1
SPEAKER_00
of a time really, just all these bacteria under constrained resources battling it out.
1
SPEAKER_00
I m sure they can invent more complex, like I don t understand, it s like how to move from a hello world program to like invent a function or something like that.
1
SPEAKER_00
I don t.
1
SPEAKER_00
So I don t, yeah, so I m with you.
1
SPEAKER_00
I just feel like I don t see any, if the origin of life, that would be my intuition, that s the hardest thing.
1
SPEAKER_00
But if that s not the hardest thing, because it happens so quickly, then it s gotta be everywhere.
1
SPEAKER_00
And yeah, maybe we re just too dumb to see it.
1
SPEAKER_01
Well, it s just, we don t have really good mechanisms for seeing this life.
1
SPEAKER_01
I mean, by what,
1
SPEAKER_01
So, I m not an expert just to preface this, but just from what I ve been looking at it.
1
SPEAKER_00
I want to meet an expert on alien intelligence and how to communicate.
1
SPEAKER_01
I m very suspicious of our ability to find these intelligences out there and to find these.
1
SPEAKER_01
Radio waves, for example, are terrible.
1
SPEAKER_01
Their power drops off as basically 1 over R squared.
1
SPEAKER_01
I remember reading that our current radio waves would not be, the ones that we are broadcasting, would not be measurable by our devices today.
1
SPEAKER_01
Only like, was it like one tenth of a light year away?
1
SPEAKER_01
Like not even, basically tiny distance because you really need like a targeted transmission of massive power directed somewhere for this to be picked up on long distances.
1
SPEAKER_01
And so I just think that our ability to measure is not amazing.
1
SPEAKER_01
I think there s probably other civilizations out there.
1
SPEAKER_01
And then the big question is, why don t they build one-man probes?
1
SPEAKER_01
And why don t they interstellar travel across the entire galaxy?
1
SPEAKER_01
And my current answer is, it s probably interstellar travel is really hard.
1
SPEAKER_01
You have the interstellar medium.
1
SPEAKER_01
If you want to move at close to the speed of light, you re going to be encountering bullets along the way.
1
SPEAKER_01
Because even tiny hydrogen atoms and little particles of dust basically have massive kinetic energy at those speeds.
1
SPEAKER_01
And so basically you need some kind of shielding.
1
SPEAKER_01
You have all the cosmic radiation.
1
SPEAKER_01
It s just like brutal out there.
1
SPEAKER_01
It s really hard.
1
SPEAKER_01
And so my thinking is maybe interstellar travel is just extremely hard.
1
SPEAKER_00
And you have to be very slow.
1
SPEAKER_00
Billions of years to build hard.
1
SPEAKER_00
It feels like we re not a billion years away from doing that.
1
SPEAKER_01
It just might be that you have to go very slowly, potentially, as an example, through space.
1
SPEAKER_00
Right, as opposed to close to the speed of light.
1
SPEAKER_01
Yeah, so I m suspicious, basically, of our ability to measure life, and I m suspicious of the ability to just permeate all of space in the galaxy or across galaxies.
1
SPEAKER_01
And that s the only way that I can currently see a way around it.
1
SPEAKER_00
Yeah, it s kind of mind-blowing to think that there s
1
SPEAKER_00
trillions of intelligent alien civilizations out there kind of slowly traveling through space.
1
SPEAKER_00
to meet each other, and some of them meet, some of them go to war, some of them collaborate.
1
SPEAKER_01
And then- Or they re all just independent.
1
SPEAKER_01
They re all just like little pockets, I don t know.
1
SPEAKER_00
Well, statistically, if there s like, if there s trillions of them, surely some of them, some of the pockets are close enough together.
1
SPEAKER_01
Some of them happen to be close, yeah.
1
SPEAKER_00
And close enough to see each other, and then once you, see, once you see something that is definitely complex life, like if we see something,
1
SPEAKER_00
we re probably going to be severely, intensely, aggressively motivated to figure out what the hell that is and try to meet them.
1
SPEAKER_00
What would be your first instinct to try to, at a generational level, meet them or defend against them?
1
SPEAKER_00
Or what would be your instinct as a president of the United States and a scientist?
1
SPEAKER_00
I don t know which hat you prefer in this question.
1
SPEAKER_01
Yeah, I think the question, it s really hard.
1
SPEAKER_01
I will say, for example, for us, we have lots of primitive life forms on Earth next to us.
1
SPEAKER_01
We have all kinds of ants and everything else, and we share space with them.
1
SPEAKER_01
We are hesitant to impact on them, and we re trying to protect them by default because they are amazing, interesting, dynamical systems that took a long time to evolve, and they are interesting and special.
1
SPEAKER_01
I don t know that you want to destroy that by default.
1
SPEAKER_01
And so I like complex dynamical systems that took a lot of time to evolve.
1
SPEAKER_01
I d like to preserve it if I can afford to.
1
SPEAKER_01
And I d like to think that the same would be true about the galactic resources and that they would think that we re kind of incredible, interesting story that took time.
1
SPEAKER_01
It took a few billion years to unravel and you don t want to just destroy it.
1
SPEAKER_00
I could see two aliens talking about Earth right now and saying, I m a big fan of complex dynamical systems.
1
SPEAKER_00
So I think it was a value to preserve these and will basically are a video game they watch or show a TV show that they watch.
1
SPEAKER_01
Yeah, I think you would need a very good reason, I think, to destroy it.
1
SPEAKER_01
Like, why don t we destroy these ant farms and so on?
1
SPEAKER_01
It s because we re not actually really in direct competition with them right now.
1
SPEAKER_01
We do it accidentally and so on, but there s plenty of resources.
1
SPEAKER_01
And so why would you destroy something that is so interesting and precious?
1
SPEAKER_00
Well, from a scientific perspective, you might probe it.
1
SPEAKER_00
You might interact with it lightly.
1
SPEAKER_00
Exactly.
1
SPEAKER_01
You might want to learn something from it, right?
1
SPEAKER_00
So I wonder, there could be certain physical phenomena that we think is a physical phenomena, but it s actually interacting with us to poke the finger and see what happens.
1
SPEAKER_01
I think it should be very interesting to scientists, other alien scientists, what happened here.
1
SPEAKER_01
And what we re seeing today is a snapshot.
1
SPEAKER_01
Basically, it s a result of a huge amount of computation.
1
SPEAKER_01
over like a billion years or something like that, so.
1
SPEAKER_00
It could have been initiated by aliens.
1
SPEAKER_00
This could be a computer running a program.
1
SPEAKER_00
Okay, if you had the power to do this, okay, for sure, at least I would, I would pick a Earth-like planet that has the conditions, based on my understanding of the chemistry prerequisites for life, and I would seed it with life and run it, right?
1
SPEAKER_00
Wouldn t you 100% do that and observe it and then protect
1
SPEAKER_00
I mean, that s not just a hell of a good TV show.
1
SPEAKER_00
It s a good scientific experiment.
1
SPEAKER_00
And it s physical simulation, right?
1
SPEAKER_00
Maybe evolution is the most, like actually running it is the most efficient way to understand computation or to compute stuff.
1
SPEAKER_01
Or to understand life or what life looks like and what branches it can take.
1
SPEAKER_00
It does make me kind of feel weird that we re a part of a science experiment, but maybe it s, everything s a science experiment.
1
SPEAKER_00
Does that change anything for us, if we re a science experiment?
1
SPEAKER_00
I don t know.
1
SPEAKER_00
Two descendants of apes talking about being inside of a science experiment.
1
SPEAKER_01
I m suspicious of this idea of a deliberate panspermia, as you described it, sort of.
1
SPEAKER_01
I don t see a divine intervention in some way in the historical record right now.
1
SPEAKER_01
I do feel like the story,
1
SPEAKER_01
in these books, like Nick Lane s books and so on, sort of makes sense.
1
SPEAKER_01
And it makes sense how life arose on Earth uniquely.
1
SPEAKER_01
And yeah, I don t need to reach for more exotic explanations right now.
1
SPEAKER_00
Sure, but NPCs inside a video game don t observe any divine intervention either.
1
SPEAKER_00
We might just be all NPCs running a kind of code.
1
SPEAKER_01
Maybe eventually they will.
1
SPEAKER_01
Currently NPCs are really dumb, but once they re running GPTs, maybe they will be like, hey, this is really suspicious.
1
SPEAKER_01
What the hell?
1
SPEAKER_00
So you famously tweeted, it looks like if you bombard Earth with photons for a while, you can emit a roadster.
1
SPEAKER_00
So if like in Hitchhiker s Guide to the Galaxy, we would summarize the story of Earth.
1
SPEAKER_00
So in that book, it s mostly harmless.
1
SPEAKER_00
What do you think is all the possible stories, like a paragraph long or a sentence long, that Earth could be summarized as?
1
SPEAKER_00
Once it s done, it s computation.
1
SPEAKER_00
So like all the possible full,
1
SPEAKER_00
If Earth is a book, right, probably there has to be an ending.
1
SPEAKER_00
I mean, there s going to be an end to Earth, and it could end in all kinds of ways.
1
SPEAKER_00
It can end soon, it can end later.
1
SPEAKER_00
What do you think are the possible stories?
1
SPEAKER_00
Well, definitely there seems to be, yeah, you re sort of,
1
SPEAKER_01
pretty incredible that these self-replicating systems will basically arise from the dynamics and then they perpetuate themselves and become more complex and eventually become conscious and build a society.
1
SPEAKER_01
I kind of feel like in some sense it s kind of like a deterministic wave that just happens on any sufficiently well-arranged system like Earth.
1
SPEAKER_01
And so I kind of feel like there s a certain sense of inevitability in it.
1
SPEAKER_01
And it s really beautiful.
1
SPEAKER_00
And it ends somehow, right?
1
SPEAKER_00
So it s a chemically diverse environment where complex dynamical systems can evolve and become further and further complex.
1
SPEAKER_00
But then there s a certain, what is it?
1
SPEAKER_00
There s certain terminating conditions.
1
SPEAKER_01
I don t know what the terminating conditions are, but definitely there s a trend line of something, and we re part of that story.
1
SPEAKER_01
Where does it go?
1
SPEAKER_01
We re famously described often as a biological bootloader for AIs, and that s because humans, we re an incredible biological system, and we re capable of computation.
1
SPEAKER_01
you know, and love and so on.
1
SPEAKER_01
But we re extremely inefficient as well.
1
SPEAKER_01
We re talking to each other through audio.
1
SPEAKER_01
It s just kind of embarrassing, honestly, that we re manipulating seven symbols serially.
1
SPEAKER_01
We re using vocal cords.
1
SPEAKER_01
It s all happening over multiple seconds.
1
SPEAKER_01
It s just kind of embarrassing when you step down to the
1
SPEAKER_01
frequencies at which computers operate or are able to operate on.
1
SPEAKER_01
Basically, it does seem like synthetic intelligences are the next stage of development.
1
SPEAKER_01
I don t know where it leads to.
1
SPEAKER_01
At some point, I suspect the universe is some kind of a puzzle, and these synthetic AIs will uncover that puzzle and solve it.
1
SPEAKER_00
And then what happens after, right?
1
SPEAKER_00
Because if you just fast forward Earth many billions of years, it s quiet, and then it s like turmoil.
1
SPEAKER_00
You see city lights and stuff like that.
1
SPEAKER_00
And then what happens at the end?
1
SPEAKER_00
Is it like a poof?
1
SPEAKER_00
Or is it like a calming?
1
SPEAKER_00
Is it explosion?
1
SPEAKER_00
Is it like Earth like a giant?
1
SPEAKER_00
Because you said emit roasters.
1
SPEAKER_00
Will it start emitting like a giant number of satellites
1
SPEAKER_01
Yes, it s some kind of a crazy explosion and we re living, we re like, we re stepping through a explosion and we re like living day to day and it doesn t look like it, but it s actually, if you, I saw a very cool animation of earth, uh, and life on earth and basically nothing happens for a long time.
1
SPEAKER_01
And then the last like two seconds, like basically cities and everything and just, and the lower orbit just gets cluttered and just the whole thing happens in the last two seconds.
1
SPEAKER_01
And you re like, this is exploding state explosion.
1
SPEAKER_00
So if you play,
1
SPEAKER_00
Yeah, yeah, if you play it at normal speed, it ll just look like an explosion.
1
SPEAKER_01
It s a firecracker.
1
SPEAKER_01
We re living in a firecracker.
1
SPEAKER_00
Where it s going to start emitting all kinds of interesting things.
1
SPEAKER_00
And then, so explosion doesn t, it might actually look like a little explosion with lights and fire and energy emitted, all that kind of stuff.
1
SPEAKER_00
But when you look inside the details of the explosion, there s actual complexity happening.
1
SPEAKER_00
where there s like, yeah, human life or some kind of life.
1
SPEAKER_00
We hope it s not a destructive firecracker.
1
SPEAKER_01
It s kind of like a constructive firecracker.
1
SPEAKER_01
All right.
1
SPEAKER_01
So given that, I think a hilarious discussion.
1
SPEAKER_01
It is really interesting to think about like what the puzzle of the universe is.
1
SPEAKER_01
Did the creator of the universe give us a message?
1
SPEAKER_01
Like, for example, in the book Contact, Carl Sagan, there s a message for humanity, for
1
SPEAKER_01
any civilization in digits in the expansion of pi in base 11 eventually, which is kind of an interesting thought.
1
SPEAKER_01
Maybe we re supposed to be giving a message to our creator.
1
SPEAKER_01
Maybe we re supposed to somehow create some kind of a quantum mechanical system that alerts them to our intelligent presence here.
1
SPEAKER_01
Because if you think about it from their perspective, it s just, say, like quantum field theory, massive cellular automaton-like thing.
1
SPEAKER_01
And how do you even notice that we exist?
1
SPEAKER_01
You might not even be able to pick us up in that simulation.
1
SPEAKER_01
And so how do you prove that you exist, that you re intelligent and that you re part of the universe?
1
SPEAKER_00
So this is like a Turing test for intelligence from Earth?
1
SPEAKER_00
Yes.
1
SPEAKER_00
Like the creator is, I mean, maybe this is like trying to complete the next word in a sentence.
1
SPEAKER_00
This is a complicated way of that.
1
SPEAKER_00
Like Earth is just, is basically sending a message back.
1
SPEAKER_01
Yeah, the puzzle is basically like alerting the creator that we exist.
1
SPEAKER_01
Or maybe the puzzle is just to just break out of the system and just, you know, stick it to the creator in some way.
1
SPEAKER_01
Basically, like if you re playing a video game, you can.
1
SPEAKER_01
you can somehow find an exploit and find a way to execute on the host machine any arbitrary code.
1
SPEAKER_01
For example, I believe someone got a game of Mario to play Pong just by exploiting it and then basically writing code and being able to execute arbitrary code in the game.
1
SPEAKER_01
And so maybe that s the puzzle, is that we should be
1
SPEAKER_01
find a way to exploit it.
1
SPEAKER_01
So I think some of these synthetic AIs will eventually find the universe to be some kind of a puzzle and then solve it in some way.
1
SPEAKER_01
And that s kind of like the end game somehow.
1
SPEAKER_00
Do you often think about it as a simulation?
1
SPEAKER_00
So, as the universe being a kind of computation that might have bugs and exploits?
1
SPEAKER_01
Yes, yeah, I think so.
1
SPEAKER_01
Is that what physics is essentially?
1
SPEAKER_01
I think it s possible that physics has exploits and we should be trying to find them.
1
SPEAKER_01
Arranging some kind of a crazy quantum mechanical system that somehow gives you buffer overflow, somehow gives you a rounding error in the floating point.
1
SPEAKER_00
Yeah, that s right.
1
SPEAKER_00
And more and more sophisticated exploits.
1
SPEAKER_00
Those are jokes, but that could be actually very close to reality.
1
SPEAKER_01
Yeah, we ll find some way to extract infinite energy.
1
SPEAKER_01
For example, when you train reinforcement learning agents in physical simulations, and you ask them to, say, run
1
SPEAKER_01
quickly on the flat ground, they ll end up doing all kinds of weird things in part of that optimization.
1
SPEAKER_01
They ll get on their back leg, and they will slide across the floor.
1
SPEAKER_01
And it s because the optimization, the enforcement learning optimization on that agent, has figured out a way to extract infinite energy from the friction forces and basically their poor implementation.
1
SPEAKER_01
And they found a way to generate infinite energy and just slide across the surface.
1
SPEAKER_01
And it s not what you expected.
1
SPEAKER_01
It s sort of like a perverse solution.
1
SPEAKER_01
And so maybe we can find something like that.
1
SPEAKER_01
Maybe we can be that little dog in this physical simulation.
1
SPEAKER_00
That cracks or escapes the intended consequences of the physics that the universe came up with.
1
SPEAKER_00
We ll figure out some kind of shortcut to some weirdness.
1
SPEAKER_00
But see, the problem with that weirdness is the first person to discover the weirdness, like sliding on the back legs,
1
SPEAKER_00
that s all we re gonna do.
1
SPEAKER_00
It very quickly becomes, everybody does that thing.
1
SPEAKER_00
So the paperclip maximizer is a ridiculous idea, but that very well could be what then we ll just all switch that, because it s so fun.
1
SPEAKER_01
Well, no person will discover it, I think, by the way.
1
SPEAKER_01
I think it s going to have to be some kind of a super intelligent AGI of a third generation.
1
SPEAKER_01
Like we re building the first generation AGI.
1
SPEAKER_00
Third generation.
1
SPEAKER_00
Yeah, so the bootloader for an AI, that AI will be a bootloader for another AI.
2
SPEAKER_01
And there s no way for us to introspect what that might have been.
2
SPEAKER_00
I think it s very likely that these things, for example, like say you have these AGIs.
2
SPEAKER_00
It s very likely, for example, they will be completely inert.
2
SPEAKER_00
I like these kinds of sci-fi books sometimes where these things are just completely inert.
2
SPEAKER_00
They don t interact with anything.
2
SPEAKER_00
And I find that kind of beautiful because they ve probably figured out the metagame of the universe in some way, potentially.
2
SPEAKER_00
They re doing something completely beyond our imagination.
2
SPEAKER_00
And they don t interact with simple chemical life forms.
2
SPEAKER_00
Like, why would you do that?
2
SPEAKER_00
So, I find those kinds of ideas compelling.
2
SPEAKER_01
What s their source of fun?
2
SPEAKER_01
What are they doing?
2
SPEAKER_01
What s the source of pleasure?
2
SPEAKER_00
Well, it s probably puzzle solving in the universe.
2
SPEAKER_01
But inert, so can you define what it means inert?
2
SPEAKER_01
So they escape the interaction with physical reality?
2
SPEAKER_00
They will appear inert to us, as in...
2
SPEAKER_00
will behave in some very strange way to us because they re playing the metagame.
2
SPEAKER_00
And the metagame is probably, say, arranging quantum mechanical systems in some very weird ways to extract infinite energy, solve the digital expansion of pi to whatever amount.
2
SPEAKER_00
They will build their own little fusion reactors or something crazy.
2
SPEAKER_00
They re doing something beyond comprehension and not understandable to us and actually brilliant under the hood.
2
SPEAKER_01
What if quantum mechanics itself is the system, and we re just thinking it s physics, but we re really parasites, not parasites, we re not really hurting physics.
2
SPEAKER_01
We re just living on this organism, and we re trying to understand it, but really it is an organism, and with a deep, deep intelligence, maybe physics itself
2
SPEAKER_01
is the organism that s doing the super interesting thing.
2
SPEAKER_01
And we re just like one little thing ant sitting on top of it trying to get energy from it.
2
SPEAKER_00
We re just kind of like these particles in the wave that I feel like is mostly deterministic and takes a universe from some kind of a Big Bang to some kind of a super intelligent replicator
2
SPEAKER_00
some kind of a stable point in the universe, given these laws of physics.
2
SPEAKER_01
You don t think, as Einstein said, God doesn t play dice?
2
SPEAKER_01
So you think it s mostly deterministic?
2
SPEAKER_01
There s no randomness in the thing?
2
SPEAKER_00
I think it s deterministic.
2
SPEAKER_00
Oh, there s tons of, well, I m gonna be careful with randomness.
2
SPEAKER_00
Pseudo-random?
2
SPEAKER_00
Yeah, I don t like random.
2
SPEAKER_00
I think maybe the laws of physics are deterministic.
2
SPEAKER_00
Yeah, I think they re deterministic.
2
SPEAKER_00
We just got really uncomfortable with this question.
2
SPEAKER_01
Do you have anxiety about whether the universe is random or not?
2
SPEAKER_01
There s no randomness.
2
SPEAKER_01
You said you like Good Will Hunting.
2
SPEAKER_01
It s not your fault, Andre.
2
SPEAKER_01
It s not your fault, man.
2
SPEAKER_00
So you don t like randomness?
2
SPEAKER_00
Yeah, I think it s unsettling.
2
SPEAKER_00
I think it s a deterministic system.
2
SPEAKER_00
I think that things that look random, like say the collapse of the wave function, et cetera, I think they re actually deterministic, just entanglement and so on.
2
SPEAKER_00
And some kind of a multi-verse theory, something, something.
2
SPEAKER_01
Okay, so why does it feel like we have a free will?
2
SPEAKER_01
Like if I raised a hand, I chose to do this now.
2
SPEAKER_01
That doesn t feel like a deterministic thing.
2
SPEAKER_01
It feels like I m making a choice.
2
SPEAKER_00
It feels like it.
2
SPEAKER_01
Okay.
2
SPEAKER_01
So it s all feelings.
2
SPEAKER_01
It s just feelings.
2
SPEAKER_01
Yeah.
2
SPEAKER_01
So when an RL agent is making a choice, is that, um, it s not really making a choice.
2
SPEAKER_01
The choices are all already there.
2
SPEAKER_00
Yeah.
2
SPEAKER_00
You re interpreting the choice and you re creating a narrative for, for having made it.
2
SPEAKER_01
Yeah, and now we re talking about the narrative, it s very meta.
2
SPEAKER_01
Looking back, what is the most beautiful or surprising idea in deep learning or AI, in general, that you ve come across?
2
SPEAKER_01
You ve seen this field explode and grow in interesting ways.
2
SPEAKER_01
What cool ideas made you sit back and go, hmm, big or small?
2
SPEAKER_00
Well, the one that I ve been thinking about recently the most probably is the
2
SPEAKER_00
transformer architecture.
2
SPEAKER_00
So basically neural networks have a lot of architectures that were trendy have come and gone for different sensory modalities like for vision, audio, text.
2
SPEAKER_00
You would process them with different looking neural nets.
2
SPEAKER_00
And recently we ve seen this convergence towards one architecture, the transformer.
2
SPEAKER_00
And you can feed it video or you can feed it images or speech or text and it just gobbles it up.
2
SPEAKER_00
And it s kind of like a
2
SPEAKER_00
bit of a general-purpose computer that is also trainable and very efficient to run on our hardware.
2
SPEAKER_00
And so this paper came out in 2016, I want to say.
2
SPEAKER_01
Attention is all you need.
2
SPEAKER_01
Attention is all you need.
2
SPEAKER_01
You criticized the paper title in retrospect, that it wasn t
2
SPEAKER_01
it didn t foresee the bigness of the impact that it was going to have.
2
SPEAKER_00
Yeah, I m not sure if the authors were aware of the impact that paper would go on to have.
2
SPEAKER_00
Probably they weren t.
2
SPEAKER_00
But I think they were aware of some of the motivations and design decisions behind the Transformer, and they chose not to, I think, expand on it in that way in the paper.
2
SPEAKER_00
And so I think they had an idea that there was more
2
SPEAKER_00
than just the surface of just like, oh, we re just doing translation and here s a better architecture.
2
SPEAKER_00
You re not just doing translation.
2
SPEAKER_00
This is like a really cool, differentiable, optimizable, efficient computer that you ve proposed.
2
SPEAKER_00
And maybe they didn t have all of that foresight, but I think it s really interesting.
2
SPEAKER_01
Isn t it funny, sorry to interrupt, that that title is memeable?
2
SPEAKER_01
That they went for such a profound idea, they went with a, I don t think anyone used that kind of title before, right?
2
SPEAKER_00
Attention is all you need.
2
SPEAKER_00
Yeah, it s like a meme or something, basically.
2
SPEAKER_01
Yeah, isn t that funny?
2
SPEAKER_01
Like, maybe if it was a more serious title, it wouldn t have the impact.
2
SPEAKER_00
Honestly, yeah, there is an element of me that honestly agrees with you and prefers it this way.
2
SPEAKER_00
Yes.
2
SPEAKER_00
If it was too grand, it would over-promise and then under-deliver, potentially.
2
SPEAKER_00
So you want to just meme your way to greatness.
2
SPEAKER_01
That should be a t-shirt.
2
SPEAKER_01
So you tweeted, the Transformer is a magnificent neural network architecture because it is a general purpose differentiable computer.
2
SPEAKER_01
It is simultaneously expressive in the forward pass, optimizable via back propagation gradient descent, and efficient.
2
SPEAKER_01
high-parallelism compute graph.
2
SPEAKER_01
Can you discuss some of those details, expressive, optimizable, efficient, for memory or in general, whatever comes to your heart?
2
SPEAKER_00
You want to have a general-purpose computer that you can train on arbitrary problems, like, say, the task of next word prediction or detecting if there s a cat in an image or something like that.
2
SPEAKER_00
And you want to train this computer, so you want to set its weights.
2
SPEAKER_00
And I think there s a number of design criteria that sort of overlap in the transformer simultaneously that made it very successful.
2
SPEAKER_00
And I think the authors were kind of deliberately trying to make this a really powerful architecture.
2
SPEAKER_00
And so basically, it s very powerful in the forward pass because it s able to express
2
SPEAKER_00
very general computation as sort of something that looks like message passing.
2
SPEAKER_00
You have nodes and they all store vectors and these nodes get to basically look at each other and it s each other s vectors and they get to communicate and basically nodes get to broadcast, hey I m looking for certain things, and then other nodes get to broadcast, hey these are the things I have, those are the keys and the values.
2
SPEAKER_00
So it s not just attention
2
SPEAKER_00
Yeah, exactly.
2
SPEAKER_00
Transformer is much more than just the attention component.
2
SPEAKER_00
It s got many pieces architecturally that went into it.
2
SPEAKER_00
The residual connection, the way it s arranged.
2
SPEAKER_00
There s a multi-layer perceptron in there, the way it s stacked and so on.
2
SPEAKER_00
But basically, there s a message-passing scheme where nodes get to look at each other, decide what s interesting, and then update each other.
2
SPEAKER_00
And so I think when you get to the details of it, I think it s a very expressive function.
2
SPEAKER_00
So it can express lots of different types of algorithms in the forward pass.
2
SPEAKER_00
Not only that, but the way it s designed with the residual connections, layer normalizations, the softmax attention and everything, it s also optimizable.
2
SPEAKER_00
This is a really big deal, because there s lots of computers that are powerful that you can t optimize, or they re not easy to optimize using the techniques that we have, which is backpropagation and gradient descent.
2
SPEAKER_00
These are first order methods, very simple optimizers, really.
2
SPEAKER_00
You also need it to be optimisable.
2
SPEAKER_00
And then lastly, you want it to run efficiently on our hardware.
2
SPEAKER_00
Our hardware is a massive throughput machine, like GPUs.
2
SPEAKER_00
They prefer lots of parallelism.
2
SPEAKER_00
So you don t want to do lots of sequential operations.
2
SPEAKER_00
You want to do a lot of operations serially.
2
SPEAKER_00
And the transformer is designed with that in mind as well.
2
SPEAKER_00
And so it s designed for our hardware, and it s designed to both be very expressive in the forward pass, but also very optimisable in the backward pass.
2
SPEAKER_01
And you said that the residual connections support a kind of ability to learn short algorithms fast and first, and then gradually extend them longer during training.
2
SPEAKER_01
What s the idea of learning short algorithms?
2
SPEAKER_00
Right.
2
SPEAKER_00
Think of it as a, so basically a transformer is a series of blocks, right?
2
SPEAKER_00
And these blocks have attention and a little multilayer perceptron.
2
SPEAKER_00
And so you go off into a block and you come back to this residual pathway, and then you go off and you come back, and then you have a number of layers arranged sequentially.
2
SPEAKER_00
And so the way to look at it, I think, is because of the residual pathway, in the backward pass, the gradients sort of flow along it uninterrupted, because addition distributes the gradient equally to all of its branches.
2
SPEAKER_00
So the gradient from the supervision at the top just floats directly to the first layer.
2
SPEAKER_00
And all the residual connections are arranged so that in the beginning, during initialization, they contribute nothing to the residual pathway.
2
SPEAKER_00
So what it kind of looks like is, imagine the transformer is kind of like a Python function, like a def, and you get to do various kinds of lines of code.
2
SPEAKER_00
Say you have a hundred layers deep transformer, typically they would be much shorter, say 20.
2
SPEAKER_00
So you have 20 lines of code and you can do something in them.
2
SPEAKER_00
And so during the optimization, basically what it looks like is first you optimize the first line of code, and then the second line of code can kick in, and the third line of code can kick in.
2
SPEAKER_00
And I kind of feel like because of the residual pathway and the dynamics of the optimization, you can sort of learn a very short algorithm that gets the approximate answer, but then the other layers can sort of kick in and start to create a contribution.
2
SPEAKER_00
And at the end of it, you re optimizing over an algorithm that is 20 lines of code.
2
SPEAKER_00
except these lines of code are very complex because it s an entire block of a transformer.
2
SPEAKER_00
You can do a lot in there.
2
SPEAKER_00
What s really interesting is that this transformer architecture actually has been remarkably resilient.
2
SPEAKER_00
Basically, the transformer that came out in 2016 is the transformer you would use today, except you reshuffle some of the layer norms.
2
SPEAKER_00
the layer normalizations have been reshuffled to a pre-norm formulation.
2
SPEAKER_00
And so it s been remarkably stable, but there s a lot of bells and whistles that people have attached on it and tried to improve it.
2
SPEAKER_00
I do think that basically it s a big step in simultaneously optimizing for lots of properties of a desirable neural network architecture.
2
SPEAKER_00
And I think people have been trying to change it, but it s proven remarkably resilient.
2
SPEAKER_00
But I do think that there should be even better architectures potentially.
2
SPEAKER_01
But you admire the resilience here.
2
SPEAKER_01
There s something profound about this architecture that leads to resilience.
2
SPEAKER_01
So maybe everything could be turned into a problem that transformers can solve.
2
SPEAKER_00
Currently, it definitely looks like the transformer is taking over AI, and you can feed basically arbitrary problems into it.
2
SPEAKER_00
And it s a general differentiable computer, and it s extremely powerful.
2
SPEAKER_00
And this convergence in AI has been really interesting to watch for me personally.
2
SPEAKER_01
what else do you think could be discovered here about transformers?
2
SPEAKER_01
Like what surprising thing?
2
SPEAKER_01
Or is it a stable, I want a stable place.
2
SPEAKER_01
Is there something interesting we might discover about transformers?
2
SPEAKER_01
Like aha moments, maybe has to do with memory, maybe knowledge representation, that kind of stuff.
2
SPEAKER_00
Definitely the Zeitgeist today is just pushing, like basically right now the Zeitgeist is do not touch the transformer, touch everything else.
2
SPEAKER_00
So people are scaling up the data sets, making them much, much bigger.
2
SPEAKER_00
They re working on the evaluation, making the evaluation much, much bigger.
2
SPEAKER_00
And they re basically keeping the architecture unchanged.
2
SPEAKER_00
And that s how we ve, that s the last five years of progress in AI, kind of.
2
SPEAKER_01
What do you think about one flavor of it, which is language models?
2
SPEAKER_01
Have you been surprised?
2
SPEAKER_01
Has your sort of imagination been captivated by, you mentioned GPT and all the bigger and bigger and bigger language models, and what are the limits of those models, do you think?
2
SPEAKER_01
So just the task of natural language.
2
SPEAKER_00
Basically, the way GPT is trained is you just download a massive amount of text data from the internet, and you try to predict the next word in the sequence, roughly speaking.
2
SPEAKER_00
You re predicting little word chunks, but roughly speaking, that s it.
2
SPEAKER_00
And what s been really interesting to watch is, basically, it s a language model.
2
SPEAKER_00
Language models have actually existed for a very long time.
2
SPEAKER_00
There s papers on language modeling from 2003, even earlier.
2
SPEAKER_01
Can you explain in that case what a language model is?
2
SPEAKER_00
Yeah.
2
SPEAKER_00
So a language model, just basically the rough idea is just predicting the next word in a sequence, roughly speaking.
2
SPEAKER_00
So there s a paper from, for example, Ben Gio and the team from 2003, where for the first time they were using a neural network to take, say, like three or five words and predict the next word.
2
SPEAKER_00
And they re doing this on much smaller data sets.
2
SPEAKER_00
And the neural net is not a transformer.
2
SPEAKER_00
It s a multi-layer perceptron.
2
SPEAKER_00
But it s the first time that a neural network has been applied in that setting.
2
SPEAKER_00
But even before neural networks, there were language models, except they were using n-gram models.
2
SPEAKER_00
So N-gram models are just count-based models.
2
SPEAKER_00
So if you start to take two words and predict the third one, you just count up how many times you ve seen any two-word combinations and what came next.
2
SPEAKER_00
And what you predict as coming next is just what you ve seen the most of in the training set.
2
SPEAKER_00
And so language modeling has been around for a long time.
2
SPEAKER_00
Neural networks have done language modeling for a long time.
2
SPEAKER_00
So really what s new or interesting or exciting is just realizing that when you scale it up,
2
SPEAKER_00
with a powerful enough neural net transformer, you have all these emergent properties where basically what happens is if you have a large enough data set of text,
2
SPEAKER_00
You are in the task of predicting the next word.
2
SPEAKER_00
You are multitasking a huge amount of different kinds of problems.
2
SPEAKER_00
You are multitasking understanding of chemistry, physics, human nature.
2
SPEAKER_00
Lots of things are sort of clustered in that objective.
2
SPEAKER_00
It s a very simple objective, but actually you have to understand a lot about the world to make that prediction.
2
SPEAKER_01
You just said the U word, understanding.
2
SPEAKER_01
Are you, in terms of chemistry and physics and so on, what do you feel like it s doing?
2
SPEAKER_01
Is it searching for the right context?
2
SPEAKER_01
What is the actual process happening here?
2
SPEAKER_00
Yeah, so basically it gets 1,000 words, and it s trying to predict 1,000 at first.
2
SPEAKER_00
And in order to do that very, very well over the entire data set available on the internet, you actually have to basically kind of understand the context of what s going on in there.
2
SPEAKER_00
And it s a sufficiently hard problem that if you have a powerful enough computer, like a transformer, you end up with interesting solutions.
2
SPEAKER_00
And you can ask it to do all kinds of things.
2
SPEAKER_00
it shows a lot of emergent properties, like in-context learning, that was the big deal with GPT and the original paper when they published it, is that you can just sort of prompt it in various ways and ask it to do various things, and it will just kind of complete the sentence.
2
SPEAKER_00
But in the process of just completing the sentence, it s actually solving all kinds of really interesting problems that we care about.
2
SPEAKER_01
Do you think it s doing something like understanding?
2
SPEAKER_01
Like when we use the word understanding for us humans,
2
SPEAKER_00
I think it s doing some understanding.
2
SPEAKER_00
In its weights, it understands, I think, a lot about the world.
2
SPEAKER_00
And it has to, in order to predict the next word in the sequence.
2
SPEAKER_01
So it s trained on the data from the internet.
2
SPEAKER_01
What do you think about this approach, in terms of data sets, of using data from the internet?
2
SPEAKER_01
Do you think the internet has enough structured data to teach AI about human civilization?
2
SPEAKER_00
Yeah, so I think the internet has a huge amount of data.
2
SPEAKER_00
I m not sure if it s a complete enough set.
2
SPEAKER_00
I don t know that text is enough for having a sufficiently powerful AGI as an outcome.
2
SPEAKER_01
Of course there is audio and video and images and all that kind of stuff.
2
SPEAKER_00
Yeah, so text by itself I m a little bit suspicious about.
2
SPEAKER_00
There s a ton of things we don t put in text in writing just because they re obvious to us about how the world works and the physics of it and that things fall.
2
SPEAKER_00
We don t put that stuff in text because why would you?
2
SPEAKER_00
We share that understanding.
2
SPEAKER_00
And so text is a communication medium between humans, and it s not an all-encompassing medium of knowledge about the world.
2
SPEAKER_00
But as you pointed out, we do have video, and we have images, and we have audio.
2
SPEAKER_00
And so I think that definitely helps a lot.
2
SPEAKER_00
But we haven t trained models sufficiently across all of those modalities yet.
2
SPEAKER_00
So I think that s what a lot of people are interested in.
2
SPEAKER_01
But I wonder what that shared understanding of what we might call common sense
2
SPEAKER_01
has to be learned, inferred in order to complete the sentence correctly.
2
SPEAKER_01
So maybe the fact that it s implied on the internet, the model s gonna have to learn that, not by reading about it, by inferring it in the representation.
2
SPEAKER_01
So like common sense, just like we, I don t think we learn common sense, like nobody says, tells us explicitly, we just figure it all out by interacting with the world.
2
SPEAKER_01
And so here s a model of reading about the way people interact with the world.
2
SPEAKER_01
I might have to infer that.
2
SPEAKER_01
I wonder.
2
SPEAKER_00
Yeah.
3
SPEAKER_01
I briefly worked on a project called World of Bits, training an RL system to take actions on the internet, versus just consuming the internet like we talked about.
3
SPEAKER_01
Do you think there s a future for that kind of system, interacting with the internet to help the learning?
3
SPEAKER_00
Yes, I think that s probably the final frontier for a lot of these models.
3
SPEAKER_00
So as you mentioned, when I was at OpenAI, I was working on this project, World of Bits, and basically it was the idea of giving neural networks access to a keyboard and a mouse.
3
SPEAKER_00
What could possibly go wrong?
3
SPEAKER_00
So basically, you perceive the input of the screen pixels.
3
SPEAKER_00
And basically, the state of the computer is sort of visualized for human consumption in images of the web browser and stuff like that.
3
SPEAKER_00
And then you give the neural network the ability to press keyboards and use the mouse.
3
SPEAKER_00
And we re trying to get it to, for example, complete bookings and interact with user interfaces.
3
SPEAKER_01
And... Where did you learn from that experience?
3
SPEAKER_01
What was some fun stuff?
3
SPEAKER_01
This is a super cool idea.
3
SPEAKER_01
Yeah.
3
SPEAKER_01
I mean, it s like...
3
SPEAKER_01
I mean, the step between observer to actor is a super fascinating step.
3
SPEAKER_00
Well, it s the universal interface in the digital realm, I would say.
3
SPEAKER_00
And there s a universal interface in the physical realm, which in my mind is a humanoid form factor kind of thing.
3
SPEAKER_00
We can later talk about Optimus and so on.
3
SPEAKER_00
But I feel like there s a
3
SPEAKER_00
they re kind of like a similar philosophy in some way, where the physical world is designed for the human form and the digital world is designed for the human form of seeing the screen and using keyboard and mouse.
3
SPEAKER_00
It s the universal interface that can basically command the digital infrastructure we ve built up for ourselves.
3
SPEAKER_00
It feels like a very powerful interface to command and to build on top of.
3
SPEAKER_00
Now, to your question as to what I learned from that, it s interesting because the world of bits was basically too early, I think, at OpenAI at the time.
3
SPEAKER_00
This is around 2015 or so, and the Zeitgeist at that time was very different in AI from the Zeitgeist today.
3
SPEAKER_00
At the time, everyone was super excited about reinforcement learning from scratch.
3
SPEAKER_00
This is the time of the Atari paper, where neural networks were playing Atari games and beating humans in some cases, AlphaGo and so on.
3
SPEAKER_00
So everyone s very excited about training neural networks from scratch using reinforcement learning directly.
3
SPEAKER_00
It turns out that reinforcement learning is extremely inefficient way of training neural networks, because you re taking all these actions and all these observations, and you get some sparse rewards once in a while.
3
SPEAKER_00
So you do all this stuff based on all these inputs.
3
SPEAKER_00
And once in a while, you re told you did a good thing.
3
SPEAKER_00
You did a bad thing.
3
SPEAKER_00
And it s just an extremely hard problem.
3
SPEAKER_00
You can t learn from that.
3
SPEAKER_00
You can burn a forest, and you can sort of brute force through it.
3
SPEAKER_00
And we saw that, I think, with Go and Dota and so on.
3
SPEAKER_00
And it does work.
3
SPEAKER_00
but it s extremely inefficient and not how you want to approach problems, practically speaking.
3
SPEAKER_00
And so that s the approach that, at the time, we also took to World of Bits.
3
SPEAKER_00
We would have an agent initialize randomly, so with keyboard mash and mouse mash, and try to make a booking.
3
SPEAKER_00
And it just revealed the insanity of that approach very quickly, where you have to stumble by the correct booking in order to get a reward of you did it correctly.
3
SPEAKER_00
And you re never going to stumble by it by chance at random.
3
SPEAKER_01
So even with a simple web interface, there s too many options.
3
SPEAKER_00
There s just too many options.
3
SPEAKER_00
And it s too sparse of a reward signal.
3
SPEAKER_00
And you re starting from scratch at the time.
3
SPEAKER_00
And so you don t know how to read.
3
SPEAKER_00
You don t understand pictures, images, buttons.
3
SPEAKER_00
You don t understand what it means to make a booking.
3
SPEAKER_00
But now what s happened is it is time to revisit that.
3
SPEAKER_00
And OpenAI is interested in this.
3
SPEAKER_00
Companies like Adept are interested in this and so on.
3
SPEAKER_00
And the idea is coming back because the interface is very powerful.
3
SPEAKER_00
But now you re not training an agent from scratch.
3
SPEAKER_00
You are taking the GPT as an initialization.
3
SPEAKER_00
So GPT is pre-trained on all of text.
3
SPEAKER_00
And it understands what s a booking.
3
SPEAKER_00
It understands what s a submit.
3
SPEAKER_00
It understands quite a bit more.
3
SPEAKER_00
And so it already has those representations.
3
SPEAKER_00
They are very powerful.
3
SPEAKER_00
And that makes all the training significantly more efficient and makes the problem tractable.
3
SPEAKER_01
Should the interaction be like the way humans see it, with the buttons and the language, or should it be with the HTML, JavaScript, and the CSS?
3
SPEAKER_01
What do you think is the better?
3
SPEAKER_00
So today, all of this interaction is mostly on the level of HTML, CSS, and so on.
3
SPEAKER_00
That s done because of computational constraints.
3
SPEAKER_00
But I think ultimately, everything is designed for human visual consumption.
3
SPEAKER_00
And so at the end of the day, there s all the additional information is in the layout of the web page and what s next to you and what s a red background and all this kind of stuff and what it looks like visually.
3
SPEAKER_00
So I think that s the final frontier as we are taking in pixels and we re giving out keyboard, mouse commands.
3
SPEAKER_00
But I think it s impractical still today.
3
SPEAKER_01
Do you worry about bots on the internet, given these ideas, given how exciting they are?
3
SPEAKER_01
Do you worry about bots on Twitter being not the stupid bots that we see now with the crypto bots, but the bots that might be out there actually that we don t see, that they re interacting in interesting ways?
3
SPEAKER_01
So this kind of system feels like it should be able to pass the I m not a robot click button, whatever.
3
SPEAKER_01
Do you actually understand how that test works?
3
SPEAKER_01
There s a checkbox or whatever that you click.
3
SPEAKER_01
It s presumably tracking mouse movement and the timing and so on.
3
SPEAKER_01
So exactly this kind of system we re talking about should be able to pass that.
3
SPEAKER_01
So what do you feel about
3
SPEAKER_01
bots that are language models plus have some interactability and are able to tweet and reply and so on, do you worry about that world?
3
SPEAKER_00
Yeah, I think it s always been a bit of an arms race between sort of the attack and the defense.
3
SPEAKER_00
So the attack will get stronger, but the defense will get stronger as well, our ability to detect that.
3
SPEAKER_01
How do you defend?
3
SPEAKER_01
How do you detect?
3
SPEAKER_01
How do you know that your Karpate account on Twitter is human?
3
SPEAKER_01
How would you approach that?
3
SPEAKER_01
Like if people were claim, you know, how would you defend yourself in the court of law that I m a human, this accountancy?
3
SPEAKER_00
Yeah, at some point I think it might be, I think the society will evolve a little bit.
3
SPEAKER_00
Like we might start signing, digitally signing some of our correspondence or, you know, things that we create.
3
SPEAKER_00
Right now it s not necessary, but maybe in the future it might be.
3
SPEAKER_00
I do think that we are going towards a world where we share the digital space with
3
SPEAKER_01
Synthetic beings.
3
SPEAKER_00
Yeah.
3
SPEAKER_00
And they will get much better and they will share our digital realm and they ll eventually share our physical realm as well.
3
SPEAKER_00
It s much harder.
3
SPEAKER_00
But that s kind of like the world we re going towards and most of them will be benign and awful and some of them will be malicious and it s going to be an arms race trying to detect them.
3
SPEAKER_01
So, I mean, the worst isn t the AIs.
3
SPEAKER_01
The worst is the AIs pretending to be human.
3
SPEAKER_01
So, I don t know if it s always malicious.
3
SPEAKER_01
There s obviously a lot of malicious applications, but it could also be, you know, if I was an AI, I would try very hard to pretend to be human because we re in a human world.
3
SPEAKER_01
I wouldn t get any respect as an AI.
3
SPEAKER_01
I wanna get some love and respect.
3
SPEAKER_00
I don t think the problem is intractable.
3
SPEAKER_00
People are thinking about the proof of personhood, and we might start digitally signing our stuff, and we might all end up having some solution for proof of personhood.
3
SPEAKER_00
It doesn t seem to me intractable.
3
SPEAKER_00
It s just something that we haven t had to do until now, but I think once the need really starts to emerge, which is soon, I think people will think about it much more.
3
SPEAKER_01
So, but that too will be a race because obviously you can probably spoof or fake the proof of personhood.
3
SPEAKER_01
So you have to try to figure out how to... Probably.
3
SPEAKER_01
I mean, it s weird that we have like social security numbers and like passports and stuff.
3
SPEAKER_01
it seems like it s harder to fake stuff in the physical space, but in the digital space, it just feels like it s gonna be very tricky, very tricky to out, because it seems to be pretty low cost to fake stuff.
3
SPEAKER_01
What, are you gonna put an AI in jail for trying to use a fake personhood proof?
3
SPEAKER_01
I mean, okay, fine, you ll put a lot of AIs in jail, but there ll be more AIs, exponentially more.
3
SPEAKER_01
The cost of creating a bot is very low.
3
SPEAKER_01
unless there s some kind of way to track accurately, you re not allowed to create any program without tying yourself to that program.
3
SPEAKER_01
Any program that runs on the internet, you ll be able to trace every single human programming that was involved with that program.
3
SPEAKER_00
maybe you have to start declaring when, you know, we have to start drawing those boundaries and keeping track of, okay, what are digital entities versus human entities, and what is the ownership of human entities and digital entities and something like that.
3
SPEAKER_00
I don t know, but I think I m optimistic that this is possible.
3
SPEAKER_00
In some sense, we re currently in the worst time of it because all these bots suddenly have become very capable, but we don t have the fences yet built up as a society.
3
SPEAKER_00
But I think that doesn t seem to me intractable.
3
SPEAKER_00
It s just something that
3
SPEAKER_01
we have to deal with.
3
SPEAKER_01
It seems weird that the Twitter bot, like really crappy Twitter bots are so numerous.
3
SPEAKER_01
So I presume that the engineers at Twitter are very good.
3
SPEAKER_01
So it seems like what I would infer from that is it seems like a hard problem.
3
SPEAKER_01
They re probably catching, all right, if I were to sort of steel man the case,
3
SPEAKER_01
It s a hard problem, and there s a huge cost to false positive, to removing a post by somebody that s not a bot.
3
SPEAKER_01
That creates a very bad user experience, so they re very cautious about removing.
3
SPEAKER_01
And maybe the bots are really good at learning what gets removed and not, such that they can stay ahead of the removal process very quickly.
3
SPEAKER_00
My impression of it, honestly, is there s a lot of low hanging fruit.
3
SPEAKER_00
I mean, it s not subtle.
3
SPEAKER_00
That s my impression of it.
3
SPEAKER_01
It s not subtle.
3
SPEAKER_01
Yeah, that s my impression as well.
3
SPEAKER_01
But it feels like maybe you re seeing the tip of the iceberg.
3
SPEAKER_01
Maybe the number of bots is in like the trillions and you have to like...
3
SPEAKER_01
just, it s a constant assault of bots.
3
SPEAKER_01
I don t know.
3
SPEAKER_01
You have to still man the case, because the bots I m seeing are pretty obvious.
3
SPEAKER_01
I could write a few lines of code that catch these bots.
3
SPEAKER_00
I mean, definitely there s a lot of longing fruit, but I will say, I agree that if you are a sophisticated actor, you could probably create a pretty good bot right now, using tools like GPTs, because it s a language model.
3
SPEAKER_00
You can generate faces that look quite good now.
3
SPEAKER_00
and you can do this at scale.
3
SPEAKER_00
And so I think, yeah, it s quite plausible and it s going to be hard to defend.
3
SPEAKER_01
There was a Google engineer that claimed that Lambda was sentient.
3
SPEAKER_01
Do you think there s any inkling of truth to what he felt?
3
SPEAKER_01
And more importantly, to me at least, do you think language models will achieve sentience or the illusion of sentience soonish?
3
SPEAKER_00
Yeah.
3
SPEAKER_00
To me, it s a little bit of a canary in a coal mine kind of moment, honestly, a little bit.
3
SPEAKER_00
So this engineer spoke to like a chatbot at Google and became convinced that this bot is sentient.
3
SPEAKER_01
Yeah, asked it some existential philosophical questions.
3
SPEAKER_00
And it gave like reasonable answers and looked real and so on.
3
SPEAKER_00
So to me, it s a
3
SPEAKER_00
He wasn t sufficiently trying to stress the system, I think, and exposing the truth of it as it is today.
3
SPEAKER_00
But I think this will be increasingly harder over time.
3
SPEAKER_00
I think more and more people will basically become
3
SPEAKER_00
Yeah, I think there will be more people like that over time as this gets better.
3
SPEAKER_01
Like form an emotional connection to an AI.
3
SPEAKER_00
Yeah, perfectly plausible in my mind.
3
SPEAKER_00
I think these AIs are actually quite good at human connection, human emotion.
3
SPEAKER_00
A ton of text on the internet is about humans and connection and love and so on.
3
SPEAKER_00
So I think they have a very good understanding in some sense of how people speak to each other about this.
3
SPEAKER_00
very capable of creating a lot of that kind of text.
3
SPEAKER_00
There s a lot of sci-fi from the 50s and 60s that imagined AIs in a very different way.
3
SPEAKER_00
They are calculating cold Vulcan-like machines.
3
SPEAKER_00
That s not what we re getting today.
3
SPEAKER_00
We re getting pretty emotional AIs that actually are very competent and capable of generating
3
SPEAKER_00
you know, plausible sounding text with respect to all these topics.
3
SPEAKER_01
See, I m really hopeful about AI systems that are like companions, that help you grow, develop as a human being, help you maximize long-term happiness.
3
SPEAKER_01
But I m also very worried about AI systems that figure out from the internet that humans get attracted to drama.
3
SPEAKER_01
And so these would just be like shit-talking AIs that just constantly, did you hear it?
3
SPEAKER_01
Like they ll do gossip, they ll try to plant seeds of suspicion to other humans that you love and trust and just kind of mess with people, you know, because that s going to get a lot of attention.
3
SPEAKER_01
So drama, maximize drama on the path to maximizing.
3
SPEAKER_01
and us humans will feed into that machine and get, it ll be a giant drama shitstorm.
3
SPEAKER_01
So I m worried about that.
3
SPEAKER_01
So it s the objective function really defines the way that human civilization progresses with AIs in it.
3
SPEAKER_00
I think right now, at least, today, it s not correct to really think of them as goal-seeking agents that want to do something.
3
SPEAKER_00
They have no long-term memory or anything.
3
SPEAKER_00
It s literally, a good approximation of it is you get 1,000 words, and you re trying to predict 1,000 at first, and then you continue feeding it in.
3
SPEAKER_00
And you are free to prompt it in whatever way you want, so in text.
3
SPEAKER_00
So you say, OK, you are a psychologist, and you are very good, and you love humans.
3
SPEAKER_00
And here s a conversation between you and another human.
3
SPEAKER_00
human colon something, you something.
3
SPEAKER_00
And then it just continues the pattern.
3
SPEAKER_00
And suddenly you re having a conversation with a fake psychologist who s like trying to help you.
3
SPEAKER_00
And so it s still kind of like in the realm of a tool.
3
SPEAKER_00
It is a, people can prompt it in arbitrary ways and it can create really incredible text, but it doesn t have long-term goals over long periods of time.
3
SPEAKER_00
It doesn t try to, so it doesn t look that way right now.
3
SPEAKER_01
Yeah, but you can do short-term goals that have long-term effects.
3
SPEAKER_01
So if my prompting
3
SPEAKER_01
short-term goal is to get Andrej Kaposic to respond to me on Twitter.
3
SPEAKER_01
I think AI might, that s the goal, but it might figure out that talking shit to you, it ll be the best in a highly sophisticated, interesting way.
3
SPEAKER_01
And then you build up a relationship when you respond once.
3
SPEAKER_01
And then over time, it gets to not be sophisticated and just talk shit.
3
SPEAKER_01
And okay, maybe it won t get to Andre, but it might get to another celebrity, it might get to other big accounts.
3
SPEAKER_01
And then it ll just, so with just that simple goal, get them to respond.
3
SPEAKER_01
Maximize the probability of actual response.
3
SPEAKER_00
Yeah, I mean, you could prompt a powerful model like this with its opinion about how to do any possible thing you re interested in.
3
SPEAKER_00
So they will just, they re kind of on track to become these oracles.
3
SPEAKER_00
I sort of think of it that way.
3
SPEAKER_00
They are oracles, currently it s just text, but they will have calculators, they will have access to Google search, they will have all kinds of gadgets and gizmos, they will be able to operate the internet and find different information.
3
SPEAKER_00
That s kind of like currently what it looks like in terms of the development.
3
SPEAKER_01
Do you think there ll be an improvement eventually over what Google is for access to human knowledge?
3
SPEAKER_01
Like it ll be a more effective search engine to access human knowledge?
3
SPEAKER_00
I think there s definite scope in building a better search engine today.
3
SPEAKER_00
And I think Google, they have all the tools, all the people, they have everything they need.
3
SPEAKER_00
They have all the puzzle pieces.
3
SPEAKER_00
They have people training transformers at scale.
3
SPEAKER_00
They have all the data.
3
SPEAKER_00
It s just not obvious if they are capable as an organization to innovate on their search engine right now.
3
SPEAKER_00
And if they don t, someone else will.
3
SPEAKER_00
There s absolute scope for building a significantly better search engine built on these tools.
3
SPEAKER_01
It s so interesting, a large company where the search, there s already an infrastructure, it works as it brings out a lot of money.
3
SPEAKER_01
So where structurally inside a company is their motivation to pivot?
3
SPEAKER_01
To say, we re going to build a new search engine.
3
SPEAKER_01
That s really hard.
3
SPEAKER_01
So it s usually going to come from a startup, right?
3
SPEAKER_00
That would be, yeah, or some other more competent organization.
3
SPEAKER_00
So I don t know.
3
SPEAKER_00
So currently, for example, maybe Bing has another shot at it, you know, as an example.
3
SPEAKER_00
There you go, Microsoft Edge, as we re talking offline.
3
SPEAKER_00
I mean, I definitely, it s really interesting because search engines used to be about, okay, here s some query, here s,
3
SPEAKER_00
here s webpages that look like the stuff that you have, but you could just directly go to answer and then have supporting evidence.
3
SPEAKER_00
These models based
4
SPEAKER_00
basically, they ve read all the texts and they ve read all the web pages.
4
SPEAKER_00
And so sometimes when you see yourself going over to search results and sort of getting like a sense of like the average answer to whatever you re interested in, like that just directly comes out, you don t have to do that work.
4
SPEAKER_00
So they re kind of like, yeah, I think they have a way to this of distilling all that knowledge into
4
SPEAKER_00
like some level of insight, basically.
4
SPEAKER_01
Do you think of prompting as a kind of teaching and learning, like this whole process, like another layer?
4
SPEAKER_01
You know, because maybe that s what humans are, where you have that background model and you re
4
SPEAKER_01
the world is prompting you.
4
SPEAKER_00
Yeah, exactly.
4
SPEAKER_00
I think the way we are programming these computers now, like GPTs, is converging to how you program humans.
4
SPEAKER_00
I mean, how do I program humans via prompt?
4
SPEAKER_00
I go to people and I prompt them to do things.
4
SPEAKER_00
I prompt them for information.
4
SPEAKER_00
And so natural language prompt is how we program humans.
4
SPEAKER_00
And we re starting to program computers directly in that interface.
4
SPEAKER_00
It s like pretty remarkable, honestly.
4
SPEAKER_01
So you ve spoken a lot about the idea of software 2.0.
4
SPEAKER_01
All good ideas become like cliches so quickly.
4
SPEAKER_01
Like the terms, it s kind of hilarious.
4
SPEAKER_01
It s like, I think Eminem once said that like, if he gets annoyed by a song he s written very quickly, that means it s gonna be a big hit.
4
SPEAKER_01
Because it s too catchy.
4
SPEAKER_01
But can you describe this idea and how you re thinking about it has evolved over the months and years since you coined it?
4
SPEAKER_00
I had a blog post on Software 2.0, I think several years ago now.
4
SPEAKER_00
The reason I wrote that post is because I saw something remarkable happening in
4
SPEAKER_00
like software development and how a lot of code was being transitioned to be written not in sort of like C++ and so on, but it s written in the weights of a neural net.
4
SPEAKER_00
Basically just saying that neural nets are taking over software, the realm of software, and taking more and more and more tasks.
4
SPEAKER_00
And at the time, I think not many people understood this deeply enough, that this is a big deal, this is a big transition.
4
SPEAKER_00
Neural networks were seen as one of multiple classification algorithms you might use for your dataset problem on Kaggle.
4
SPEAKER_00
This is not that.
4
SPEAKER_00
This is a change in how we program computers.
4
SPEAKER_00
And I saw neural nets as this is going to take over.
4
SPEAKER_00
The way we program computers is going to change.
4
SPEAKER_00
It s not going to be people writing software in C++ or something like that and directly programming the software.
4
SPEAKER_00
It s going to be accumulating training sets and data sets and crafting these objectives by which we train these neural nets.
4
SPEAKER_00
And at some point, there s going to be a compilation process from the datasets and the objective and the architecture specification into the binary, which is really just the neural net weights and the forward pass of the neural net.
4
SPEAKER_00
And then you can deploy that binary.
4
SPEAKER_00
And so I was talking about that sort of transition, and that s what the post is about.
4
SPEAKER_00
And I saw this sort of play out in a lot of fields, Autopilot being one of them, but also just simple image classification.
4
SPEAKER_00
People thought originally, in the 80s and so on, that they would write the algorithm for detecting a dog in an image.
4
SPEAKER_00
And they had all these ideas about how the brain does it.
4
SPEAKER_00
And first we detect corners, and then we detect lines, and then we stitched them up.
4
SPEAKER_00
And they were really going at it.
4
SPEAKER_00
They were thinking about how they re going to write the algorithm.
4
SPEAKER_00
And this is not the way you build it.
4
SPEAKER_00
And there was a smooth transition where, OK, first we thought we were going to build everything.
4
SPEAKER_00
Then we were building the features, so like hog features and things like that, that detect these little statistical patterns from image patches.
4
SPEAKER_00
And then there was a little bit of learning on top of it, like a support vector machine or binary classifier for cat versus dog and images on top of the features.
4
SPEAKER_00
So we wrote the features, but we trained
4
SPEAKER_00
the last layer, sort of the classifier.
4
SPEAKER_00
And then people are like, actually, let s not even design the features, because we can t.
4
SPEAKER_00
Honestly, we re not very good at it.
4
SPEAKER_00
So let s also learn the features.
4
SPEAKER_00
And then you end up with basically a convolutional neural net, where you re learning most of it.
4
SPEAKER_00
You re just specifying the architecture, and the architecture has tons of fill-in-the-blanks, which is all the knobs, and you let the optimization write most of it.
4
SPEAKER_00
And so this transition is happening across the industry everywhere.
4
SPEAKER_00
And suddenly we end up with a ton of code that is written in neural netweights.
4
SPEAKER_00
And I was just pointing out that the analogy is actually pretty strong.
4
SPEAKER_00
And we have a lot of developer environments for software 1.0.
4
SPEAKER_00
Like we have IDEs, how you work with code, how you debug code, how you run code, how do you maintain code.
4
SPEAKER_00
We have GitHub.
4
SPEAKER_00
So I was trying to make those analogies in the new realm.
4
SPEAKER_00
Like what is the GitHub of software 2.0?
4
SPEAKER_00
Turns out it s something that looks like Hugging Face right now.
4
SPEAKER_00
And so I think some people took it seriously and built cool companies.
4
SPEAKER_00
And many people originally attacked the post.
4
SPEAKER_00
It actually was not well received when I wrote it.
4
SPEAKER_00
And I think maybe it has something to do with the title, but the post was not well received.
4
SPEAKER_00
And I think more people sort of have been coming around to it over time.
4
SPEAKER_01
Yeah, so you were the director of AI at Tesla, where I think this idea was really implemented at scale, which is how you have engineering teams doing software 2.0.
4
SPEAKER_01
So can you sort of linger on that idea of, I think we re in the really early stages of everything you just said, which is like GitHub IDEs, like how do we build engineering teams that
4
SPEAKER_01
that work in software 2.0 systems, and the data collection and the data annotation, which is all part of that software 2.0.
4
SPEAKER_01
Like, what do you think is the task of programming a software 2.0?
4
SPEAKER_01
Is it debugging in the space of hyperparameters, or is it also debugging in the space of data?
4
SPEAKER_00
Yeah.
4
SPEAKER_00
The way by which you program the computer and influence its algorithm is not by writing the commands yourself.
4
SPEAKER_00
You re changing mostly the dataset.
4
SPEAKER_00
You re changing the loss functions of what the neural net is trying to do, how it s trying to predict things.
4
SPEAKER_00
But yeah, basically the datasets and the architectures of the neural net.
4
SPEAKER_00
So in the case of the autopilot, a lot of the data sets had to do with, for example, detection of objects and lane line markings and traffic lights and so on.
4
SPEAKER_00
So you accumulate massive data sets of, here s an example, here s the desired label, and then here s roughly what the algorithm should look like, and that s a convolutional neural net.
4
SPEAKER_00
So the specification of the architecture is like a hint as to what the algorithm should roughly look like.
4
SPEAKER_00
And then the fill-in-the-blanks process of optimization is the training process.
4
SPEAKER_00
and then you take your neural net that was trained, it gives all the right answers on your dataset, and you deploy it.
4
SPEAKER_01
So there s, in that case, perhaps in all machine learning cases, there s a lot of tasks.
4
SPEAKER_01
So is coming up, formulating a task, like for a multi-headed neural network, is formulating a task part of the programming?
4
SPEAKER_00
Yeah, very much so.
4
SPEAKER_01
How do you break down a problem into a set of tasks?
4
SPEAKER_00
Yeah.
4
SPEAKER_00
On a high level, I would say, if you look at the software running in the autopilot, I gave a number of talks on this topic.
4
SPEAKER_00
I would say, originally, a lot of it was written in software 1.0.
4
SPEAKER_00
Imagine lots of C++.
4
SPEAKER_00
And then gradually, there was a tiny neural net that was, for example, predicting, given a single image, is there a traffic light or not, or is there a landline marking or not?
4
SPEAKER_00
And this neural net didn t have too much to do in the scope of the software.
4
SPEAKER_00
It was making tiny predictions on individual little image.
4
SPEAKER_00
And then the rest of the system stitched it up.
4
SPEAKER_00
So OK, we don t have just a single camera.
4
SPEAKER_00
We have eight cameras.
4
SPEAKER_00
We actually have eight cameras over time.
4
SPEAKER_00
And so what do you do with these predictions?
4
SPEAKER_00
How do you put them together?
4
SPEAKER_00
How do you do the fusion of all that information?
4
SPEAKER_00
And how do you act on it?
4
SPEAKER_00
All of that was written by humans in C++.
4
SPEAKER_00
And then we decided, OK, we don t actually want to do all of that fusion in C++ code, because we re actually not good enough to write that algorithm.
4
SPEAKER_00
We want the neural nets to write the algorithm.
4
SPEAKER_00
And we want to port all of that software into the 2.0 stack.
4
SPEAKER_00
And so then we actually had neural nets that now take all the eight camera images simultaneously and make predictions for all of that.
4
SPEAKER_00
And actually, they don t make predictions in the space of images.
4
SPEAKER_00
They now make predictions directly in 3D.
4
SPEAKER_00
And actually, they don t in three dimensions around the car.
4
SPEAKER_00
And now, actually, we don t manually fuse the predictions in 3D over time.
4
SPEAKER_00
We don t trust ourselves to write that tracker.
4
SPEAKER_00
So actually, we give the neural net the information over time.
4
SPEAKER_00
So it takes these videos now and makes those predictions.
4
SPEAKER_00
And so you re sort of just like putting more and more power into the neural network, more processing.
4
SPEAKER_00
And at the end of it, the eventual sort of goal is to have most of the software potentially be in the 2.0 land because it works significantly better.
4
SPEAKER_00
Humans are just not very good at writing software, basically.
4
SPEAKER_01
So the prediction is happening in this like 4D land with three-dimensional world over time.
4
SPEAKER_01
How do you...
4
SPEAKER_01
do annotation in that world?
4
SPEAKER_01
So data annotation, whether it s self-supervised or manual by humans, is a big part of this software 2.0 world.
4
SPEAKER_01
Right.
4
SPEAKER_00
I would say by far in the industry, if you re talking about the industry and what is the technology of what we have available, everything is supervised learning.
4
SPEAKER_00
So you need data sets of input, desired output, and you need lots of it.
4
SPEAKER_00
And there are three properties of it that you need.
4
SPEAKER_00
You need it to be very large.
4
SPEAKER_00
You need it to be accurate, no mistakes.
4
SPEAKER_00
And you need it to be diverse.
4
SPEAKER_00
You don t want to just have a lot of correct examples of one thing.
4
SPEAKER_00
You need to really cover the space of possibility as much as you can.
4
SPEAKER_00
And the more you can cover the space of possible inputs, the better the algorithm will work at the end.
4
SPEAKER_00
Now, once you have really good data sets that you re collecting, curating, and cleaning, you can train your neural net
4
SPEAKER_00
on top of that.
4
SPEAKER_00
So a lot of the work goes into cleaning those datasets.
4
SPEAKER_00
Now, as you pointed out, it s probably, it could be, the question is, how do you achieve a ton of, if you want to basically predict in 3D, you need data in 3D to back that up.
4
SPEAKER_00
So in this video, we have eight videos coming from all the cameras of the system.
4
SPEAKER_00
And this is what they saw, and this is the truth of what actually was around.
4
SPEAKER_00
There was this car, there was this car, this car.
4
SPEAKER_00
These are the lane line markings.
4
SPEAKER_00
This is the geometry of the road.
4
SPEAKER_00
There s a traffic light in this three-dimensional position.
4
SPEAKER_00
You need the ground truth.
4
SPEAKER_00
And so the big question that the team was solving, of course, is how do you arrive at that ground truth?
4
SPEAKER_00
Because once you have a million of it, and it s large, clean, and diverse, then training a neural net on it works extremely well, and you can ship that into the car.
4
SPEAKER_00
And so there s many mechanisms by which we collected that training data.
4
SPEAKER_00
You can always go for human annotation.
4
SPEAKER_00
You can go for simulation as a source of ground truth.
4
SPEAKER_00
You can also go for what we call the offline tracker.
4
SPEAKER_00
that we ve spoken about at the AI day and so on, which is basically an automatic reconstruction process for taking those videos and recovering the three-dimensional sort of reality of what was around that car.
4
SPEAKER_00
So basically think of doing like a three-dimensional reconstruction as an offline thing, and then understanding that, okay, there s 10 seconds of video, this is what we saw, and therefore, here s all the lane lines cars and so on.
4
SPEAKER_00
And then once you have that annotation, you can train neural nets to imitate it.
4
SPEAKER_01
And how difficult is the reconstruction?
4
SPEAKER_00
It s difficult, but it can be done.
4
SPEAKER_01
So there s overlap between the cameras and you do the reconstruction and there s perhaps if there s any inaccuracy, so that s caught in the annotation step.
4
SPEAKER_00
Yes, the nice thing about the annotation is that it is fully offline, you have infinite time, you have a chunk of one minute, and you re trying to just offline in a supercomputer somewhere, figure out where were the positions of all the cars, all the people, and you have your full one minute of video from all the angles, and you can run all the neural nets you want, and they can be very efficient, massive neural nets.
4
SPEAKER_00
There can be neural nets that can t even run in the car later at test time.
4
SPEAKER_00
so they can be even more powerful neural nets than what you can eventually deploy.
4
SPEAKER_00
So you can do anything you want, three-dimensional reconstruction, neural nets, anything you want just to recover that truth, and then you supervise that truth.
4
SPEAKER_01
What have you learned, you said no mistakes, about humans doing annotation?
4
SPEAKER_01
Because I assume humans are, there s like a range of things they re good at in terms of clicking stuff on screen.
4
SPEAKER_01
How interesting is that to you of a problem of designing an annotator where humans are accurate, enjoy it?
4
SPEAKER_01
Like what are even the metrics?
4
SPEAKER_01
Are efficient, are productive, all that kind of stuff?
4
SPEAKER_00
Yeah, so I grew the annotation team at Tesla from basically zero to 1,000 while I was there.
4
SPEAKER_00
That was really interesting.
4
SPEAKER_00
My background is a PhD student researcher, so growing that kind of an organization was pretty crazy.
4
SPEAKER_00
But yeah, I think it s extremely interesting and part of the design process very much behind the autopilot as to where you use humans.
4
SPEAKER_00
Humans are very good at certain kinds of annotations.
4
SPEAKER_00
They re very good, for example, at two-dimensional annotations of images.
4
SPEAKER_00
They re not good at annotating
4
SPEAKER_00
cars over time in three-dimensional space.
4
SPEAKER_00
Very, very hard.
4
SPEAKER_00
And so that s why we were very careful to design the tasks that are easy to do for humans versus things that should be left to the offline tracker.
4
SPEAKER_00
Like maybe the computer will do all the triangulation and 3D reconstruction, but the human will say exactly these pixels of the image are a car.
4
SPEAKER_00
Exactly these pixels are a human.
4
SPEAKER_00
And so co-designing the data annotation pipeline was very much bread and butter was what I was doing daily.
4
SPEAKER_01
Do you think there s still a lot of open problems in that space?
4
SPEAKER_01
Just in general, annotation, where the stuff the machines are good at, machines do, and the humans do what they re good at, and there s maybe some iterative process.
4
SPEAKER_00
I think to a very large extent, we went through a number of iterations, and we learned a ton about how to create these data sets.
4
SPEAKER_00
I m not seeing big open problems.
4
SPEAKER_00
Originally, when I joined, I was really not sure how this would turn out.
4
SPEAKER_00
But by the time I left, I was much more secure.
4
SPEAKER_00
And actually, we sort of understand the philosophy of how to create these data sets.
4
SPEAKER_00
And I was pretty comfortable with where that was at the time.
4
SPEAKER_01
So what are strengths and limitations of cameras for the driving task?
4
SPEAKER_01
In your understanding, when you formulate the driving task as a vision task with eight cameras, you ve seen that the entire, you know, most of the history of the computer vision field, when it has to do with neural networks, what, just if you step back, what are the strengths and limitations of pixels, of using pixels to drive?
4
SPEAKER_00
Yeah, pixels, I think, are a beautiful sensor, I would say.
4
SPEAKER_00
The thing is, cameras are very, very cheap, and they provide a ton of information, a ton of bits.
4
SPEAKER_00
So it s an extremely cheap sensor for a ton of bits, and each one of these bits has a constraint on the state of the world.
4
SPEAKER_00
And so you get lots of megapixel images, very cheap, and it just gives you all these constraints for understanding what s actually out there in the world.
4
SPEAKER_00
So Vision is probably the highest bandwidth sensor
4
SPEAKER_00
It s a very high bandwidth sensor.
4
SPEAKER_01
I love that pixels is a constraint on the world.
4
SPEAKER_01
It s this highly complex, high bandwidth constraint on the world, on the state of the world.
4
SPEAKER_00
That s fascinating.
4
SPEAKER_00
It s not just that, but again, this real importance of
4
SPEAKER_00
the sensor that humans use.
4
SPEAKER_00
Therefore, everything is designed for that sensor.
4
SPEAKER_00
The text, the writing, the flashing signs, everything is designed for vision.
4
SPEAKER_00
And so you just find it everywhere.
4
SPEAKER_00
And so that s why that is the interface you want to be in, talking again about these universal interfaces.
4
SPEAKER_00
And that s where we actually want to measure the world as well and then develop software for that sensor.
4
SPEAKER_01
But there s other constraints on the state of the world that humans use to understand the world.
4
SPEAKER_01
I mean, vision ultimately is the main one, but we re like referencing our understanding of human behavior and some common sense physics that could be inferred from vision from a perception perspective, but it feels like we re using some kind of reasoning
4
SPEAKER_01
to predict the world, not just the pixels.
4
SPEAKER_00
I mean, you have a powerful prior for how the world evolves over time, etc.
4
SPEAKER_00
So it s not just about the likelihood term coming up from the data itself telling you about what you are observing, but also the prior term of where are the likely things to see and how do they likely move and so on.
4
SPEAKER_01
And the question is how complex is the range of possibilities that might happen in the driving task.
4
SPEAKER_01
Is that to you still an open problem of how difficult is driving, like philosophically speaking?
4
SPEAKER_01
All the time you worked on driving, do you understand how hard driving is?
4
SPEAKER_00
Yeah, driving is really hard.
4
SPEAKER_00
because it has to do with the predictions of all these other agents and the theory of mind and what they re going to do.
4
SPEAKER_00
Are they looking at you?
4
SPEAKER_00
Where are they looking?
4
SPEAKER_00
Where are they thinking?
4
SPEAKER_00
There s a lot that goes there at the full tail of the expansion of the nines that we have to be comfortable with eventually.
4
SPEAKER_00
the final problems are of that form.
4
SPEAKER_00
I don t think those are the problems that are very common.
4
SPEAKER_00
I think eventually they re important, but it s like really in the tail end.
4
SPEAKER_01
In the tail end, the rare edge cases.
4
SPEAKER_01
From the vision perspective, what are the toughest parts of the vision problem of driving?
4
SPEAKER_00
Well, basically, the sensor is extremely powerful, but you
5
SPEAKER_00
you still need to process that information.
5
SPEAKER_00
And so going from brightnesses of these pixel values to, hey, here are the three-dimensional world, is extremely hard.
5
SPEAKER_00
And that s what the neural networks are fundamentally doing.
5
SPEAKER_00
And so the difficulty really is in just doing an extremely good job of engineering the entire pipeline, the entire data engine, having the capacity to train these neural nets, having the ability to evaluate the system and iterate on it.
5
SPEAKER_00
So I would say just doing this in production at scale is like the hard part.
5
SPEAKER_00
It s an execution problem.
5
SPEAKER_01
So the data engine, but also the sort of deployment of the system such that it has low latency performance, so it has to do all these steps.
5
SPEAKER_00
Yeah, for the neural net specifically, just making sure everything fits into the chip on the car.
5
SPEAKER_00
and you have a finite budget of flops that you can perform, and memory bandwidth and other constraints, and you have to make sure it flies, and you can squeeze in as much compute as you can into the tiny.
5
SPEAKER_01
What have you learned from that process?
5
SPEAKER_01
Because maybe that s one of the bigger, like, new things coming from a research background, where there s a system that has to run under heavily constrained resources, has to run really fast.
5
SPEAKER_01
What kind of insights have you learned from that?
5
SPEAKER_00
Yeah, I m not sure if there s too many insights.
5
SPEAKER_00
You re trying to create a neural net that will fit in what you have available, and you re always trying to optimize it.
5
SPEAKER_00
And we talked a lot about it on AI Day and basically the triple backflips that the team is doing to make sure it all fits and utilizes the engine.
5
SPEAKER_00
So I think it s extremely good engineering.
5
SPEAKER_00
And then there s all kinds of little insights peppered in on how to do it properly.
5
SPEAKER_01
Let s actually zoom out, because I don t think we talked about the data engine, the entirety of the layout of this idea that I think is just beautiful, with humans in the loop.
5
SPEAKER_01
Can you describe the data engine?
5
SPEAKER_00
The data engine is what I call the almost biological feeling like process by which you perfect the training sets for these neural networks.
5
SPEAKER_00
So because most of the programming now is in the level of these data sets and make sure they re large, diverse, and clean, basically you have a data set that you think is good.
5
SPEAKER_00
You train your neural net, you deploy it, and then you observe how well it s performing.
5
SPEAKER_00
And you re trying to always increase the quality of your dataset.
5
SPEAKER_00
So you re trying to catch scenarios that are basically rare.
5
SPEAKER_00
And it is in these scenarios that neural nets will typically struggle in because they weren t told what to do in those rare cases in the dataset.
5
SPEAKER_00
But now you can close the loop because if you can now collect all those at scale, you can then feed them back into the reconstruction process I described and reconstruct the truth in those cases and add it to the dataset.
5
SPEAKER_00
And so the whole thing ends up being like a staircase of improvement, of perfecting your training set.
5
SPEAKER_00
And you have to go through deployments so that you can mine the parts that are not yet represented well in the dataset.
5
SPEAKER_00
So your data set is basically imperfect.
5
SPEAKER_00
It needs to be diverse.
5
SPEAKER_00
It has pockets that are missing, and you need to pad out the pockets.
5
SPEAKER_00
You can sort of think of it that way.
5
SPEAKER_00
in the data.
5
SPEAKER_01
What role do humans play in this?
5
SPEAKER_01
So what s this biological system, like are human bodies made up of cells?
5
SPEAKER_01
What role, like how do you optimize the human system?
5
SPEAKER_01
The multiple engineers collaborating, figuring out what to focus on, what to contribute, which task to optimize in this neural network?
5
SPEAKER_01
Who s in charge of figuring out which task needs more data?
5
SPEAKER_01
Can you speak to the hyperparameters of the human system?
5
SPEAKER_00
It really just comes down to extremely good execution from an engineering team who knows what they re doing.
5
SPEAKER_00
They understand intuitively the philosophical insights underlying the data engine and the process by which the system improves.
5
SPEAKER_00
and how to, again, delegate the strategy of the data collection, how that works, and then just making sure it s all extremely well executed.
5
SPEAKER_00
And that s where most of the work is.
5
SPEAKER_00
It s not even the philosophizing or the research or the ideas of it.
5
SPEAKER_00
It s just extremely good execution.
5
SPEAKER_00
It s so hard when you re dealing with data at that scale.
5
SPEAKER_01
So your role in the data engine, executing well on it, is difficult and extremely important.
5
SPEAKER_01
Is there a priority of like a vision board of saying like, we really need to get better at stoplights?
5
SPEAKER_01
Like the prioritization of tasks.
5
SPEAKER_01
Is that essentially, and that comes from the data.
5
SPEAKER_00
That comes to a very large extent to what we are trying to achieve in the product roadmap, what we re trying to, the release we re trying to get out, and the feedback from the QA team where the system is struggling or not, the things that we re trying to improve.
5
SPEAKER_01
And the QA team gives some signal, some information in aggregate about the performance of the system in various conditions.
5
SPEAKER_00
That s right.
5
SPEAKER_00
And then, of course, all of us drive it, and we can also see it.
5
SPEAKER_00
It s really nice to work with a system that you can also experience yourself, and you know, it drives you home.
5
SPEAKER_01
Is there some insight you can draw from your individual experience that you just can t quite get from an aggregate statistical analysis of the data?
5
SPEAKER_01
It s so weird, right?
5
SPEAKER_01
It s not scientific in a sense, because you re just one anecdotal sample.
5
SPEAKER_00
Yeah, I think there s a ton of, it s a source of truth.
5
SPEAKER_00
It s your interaction with the system and you can see it, you can play with it, you can perturb it, you can get a sense of it, you have an intuition for it.
5
SPEAKER_00
I think numbers just like have a way of, numbers and plots and graphs are much harder.
5
SPEAKER_01
It hides a lot of- It s like if you train a language model, it s a really powerful way is by you interacting with it.
5
SPEAKER_01
Yeah, 100%.
5
SPEAKER_01
To try to build up an intuition.
5
SPEAKER_00
Yeah, I think Elon also, he always wanted to drive the system himself.
5
SPEAKER_00
He drives a lot, and I want to say almost daily.
5
SPEAKER_00
So he also sees this as a source of truth, you driving the system and it performing.
5
SPEAKER_01
So what do you think?
5
SPEAKER_01
Tough questions here.
5
SPEAKER_01
So Tesla last year removed radar from
5
SPEAKER_01
from the sensor suite and now just announced that it s going to remove all ultrasonic sensors relying solely on vision, so camera only.
5
SPEAKER_01
Does that make the perception problem harder or easier?
5
SPEAKER_00
I would almost reframe the question in some way.
5
SPEAKER_00
So the thing is basically, you would think that additional sensors- By the way, can I just interrupt?
5
SPEAKER_01
Go ahead.
5
SPEAKER_01
I wonder if a language model will ever do that if you prompt it.
5
SPEAKER_01
Let me reframe your question.
5
SPEAKER_01
That would be epic.
5
SPEAKER_01
This is the wrong prompt.
5
SPEAKER_01
Sorry.
5
SPEAKER_00
It s like a little bit of a wrong question because basically you would think that these sensors are an asset to you.
5
SPEAKER_01
Yeah.
5
SPEAKER_00
But if you fully consider the entire product in its entirety,
5
SPEAKER_00
These sensors are actually potentially a liability because these sensors aren t free.
5
SPEAKER_00
They don t just appear on your car.
5
SPEAKER_00
Suddenly you have an entire supply chain.
5
SPEAKER_00
You have people procuring it.
5
SPEAKER_00
There can be problems with them.
5
SPEAKER_00
They may need replacement.
5
SPEAKER_00
They are part of the manufacturing process.
5
SPEAKER_00
They can hold back the line in production.
5
SPEAKER_00
You need to source them, you need to maintain them, you have to have teams that write the firmware, all of it.
5
SPEAKER_00
And then you also have to incorporate and fuse them into the system in some way.
5
SPEAKER_00
And so it actually like bloats a lot of it.
5
SPEAKER_00
And I think Elon is really good at simplify, simplify, best part is no part.
5
SPEAKER_00
And he always tries to throw away things that are not essential because he understands the entropy in organizations and in approach.
5
SPEAKER_00
And I think in this case,
5
SPEAKER_00
the cost is high, and you re not potentially seeing it if you re just a computer vision engineer, and I m just trying to improve my network, and is it more useful or less useful?
5
SPEAKER_00
How useful is it?
5
SPEAKER_00
And the thing is, once you consider the full cost of a sensor, it actually is potentially a liability, and you need to be really sure that it s giving you extremely useful information.
5
SPEAKER_00
In this case, we looked at using it or not using it, and the delta was not massive, and so it s not useful.
5
SPEAKER_01
Is it also blow in the data engine, like having more sensors? 100%.
5
SPEAKER_01
It s a distraction.
5
SPEAKER_00
And these sensors, they can change over time, for example.
5
SPEAKER_00
You can have one type of, say, radar.
5
SPEAKER_00
You can have other type of radar.
5
SPEAKER_00
They change over time.
5
SPEAKER_00
Now you suddenly need to worry about it.
5
SPEAKER_00
Now suddenly you have a column in your SQLite telling you, oh, what sensor type was it?
5
SPEAKER_00
And they all have different distributions.
5
SPEAKER_00
And then they contribute noise and entropy into everything.
5
SPEAKER_00
And they bloat stuff.
5
SPEAKER_00
And also, organizationally, it s been really fascinating to me that it can be very distracting.
5
SPEAKER_00
If all you want to get to work is vision, all the resources are on it, and you re building out a data engine, and you re actually making forward progress, because that is the sensor with the most bandwidth, the most constraints on the world, and you re investing fully into that, and you can make that extremely good.
5
SPEAKER_00
You have only a finite amount of spend of focus across different facets of the system.
5
SPEAKER_01
And this kind of reminds me of Rich Sutton s The Bitter Lesson, that it just seems like simplifying the system in the long run, now of course you don t know what the long run is, and it seems to be always the right solution.
5
SPEAKER_01
In that case it was for RL, but it seems to apply generally across all systems that do computation.
5
SPEAKER_01
So what do you think about the LiDAR as a crutch debate, the battle between point clouds and pixels?
5
SPEAKER_00
Yeah, I think this debate is always like slightly confusing to me because it seems like the actual debate should be about like, do you have the fleet or not?
5
SPEAKER_00
That s like the really important thing about whether you can achieve a really good functioning of an AI system at this scale.
5
SPEAKER_00
So data collection systems.
5
SPEAKER_00
Yeah.
5
SPEAKER_00
Do you have a fleet or not?
5
SPEAKER_00
It s significantly more important whether you have LiDAR or not.
5
SPEAKER_00
It s just another sensor.
5
SPEAKER_00
And yeah, I think similar to the radar discussion, basically,
5
SPEAKER_00
I don t think it basically doesn t offer extra information.
5
SPEAKER_00
It s extremely costly.
5
SPEAKER_00
It has all kinds of problems.
5
SPEAKER_00
You have to worry about it.
5
SPEAKER_00
You have to calibrate it, et cetera.
5
SPEAKER_00
It creates bloat and entropy.
5
SPEAKER_00
You have to be really sure that you need this sensor.
5
SPEAKER_00
In this case, I basically don t think you need it.
5
SPEAKER_00
And I think, honestly, I will make a stronger statement.
5
SPEAKER_00
I think some of the other companies that are using it are probably going to drop it.
5
SPEAKER_01
Yeah, so you have to consider the sensor in the full, in considering can you build a big fleet that collects a lot of data, and can you integrate that sensor with that data and that sensor into a data engine that s able to quickly find different parts of the data that then continuously improves whatever the model that you re using.
5
SPEAKER_00
Yeah, another way to look at it is like vision is necessary in a sense that the world is designed for human visual consumption.
5
SPEAKER_00
So you need vision.
5
SPEAKER_00
It s necessary.
5
SPEAKER_00
And then also it is sufficient because it has all the information that you need for driving.
5
SPEAKER_00
And humans, obviously, have a vision to drive.
5
SPEAKER_00
So it s both necessary and sufficient.
5
SPEAKER_00
So you want to focus resources.
5
SPEAKER_00
And you have to be really sure if you re going to bring in other sensors.
5
SPEAKER_00
You could add sensors to infinity.
5
SPEAKER_00
At some point, you need to draw the line.
5
SPEAKER_00
And I think in this case, you have to really consider the full cost of any one sensor that you re adopting, and do you really need it?
5
SPEAKER_00
And I think the answer in this case is no.
5
SPEAKER_01
So what do you think about the idea that the other companies are forming high-resolution maps and constraining heavily the geographic regions in which they operate?
5
SPEAKER_01
Is that approach, in your view, not going to scale over time to the entirety of the United States?
5
SPEAKER_00
I think as you mentioned, like they pre-map all the environments and they need to refresh the map and they have a perfect centimeter level accuracy map of everywhere they re going to drive.
5
SPEAKER_00
It s crazy.
5
SPEAKER_00
How are you going to, when we re talking about the autonomy actually changing the world, we re talking about the deployment.
5
SPEAKER_00
on a global scale of autonomous systems for transportation.
5
SPEAKER_00
And if you need to maintain a centimeter accurate map for Earth, or like for many cities and keep them updated, it s a huge dependency that you re taking on, huge dependency.
5
SPEAKER_00
It s a massive, massive dependency.
5
SPEAKER_00
And now you need to ask yourself, do you really need it?
5
SPEAKER_00
And humans don t need it, right?
5
SPEAKER_00
So it s very useful to have a low-level map of like, OK, the connectivity of your road.
5
SPEAKER_00
You know that there s a fork coming up.
5
SPEAKER_00
When you drive an environment, you sort of have that high-level understanding.
5
SPEAKER_00
It s like a small Google Map.
5
SPEAKER_00
And Tesla uses Google Map, like similar kind of resolution information in its system.
5
SPEAKER_00
But it will not pre-map environments to semi-level accuracy.
5
SPEAKER_00
It s a crutch.
5
SPEAKER_00
It s a distraction.
5
SPEAKER_00
It costs entropy.
5
SPEAKER_00
And it diffuses the team.
5
SPEAKER_00
It dilutes the team.
5
SPEAKER_00
And you re not focusing on what s actually necessary, which is?
5
SPEAKER_00
computer vision problem.
5
SPEAKER_01
What did you learn about machine learning, about engineering, about life, about yourself as one human being from working with Elon Musk?
5
SPEAKER_00
I think the most I ve learned is about how to sort of run organizations efficiently and how to create efficient organizations and how to fight entropy in an organization.
5
SPEAKER_00
So human engineering in the fight against entropy.
5
SPEAKER_00
Yeah.
5
SPEAKER_00
I think Elon is a very efficient warrior in the fight against entropy in organizations.
5
SPEAKER_01
What does entropy in an organization look like exactly?
5
SPEAKER_00
It s process.
5
SPEAKER_00
its process and inefficiencies in the form of meetings and that kind of stuff.
5
SPEAKER_00
Yeah, meetings.
5
SPEAKER_00
He hates meetings.
5
SPEAKER_00
He keeps telling people to skip meetings if they re not useful.
5
SPEAKER_00
He basically runs the world s biggest startups, I would say.
5
SPEAKER_00
Tesla, SpaceX are the world s biggest startups.
5
SPEAKER_00
Tesla actually has multiple startups.
5
SPEAKER_00
I think it s better to look at it that way.
5
SPEAKER_00
And so I think he s extremely good at that.
5
SPEAKER_00
And yeah, he has a very good intuition for streamlining processes, making everything efficient.
5
SPEAKER_00
Best part is no part, simplifying, focusing, and just kind of removing barriers, moving very quickly, making big moves.
5
SPEAKER_00
All of this is a very startup-y sort of seeming things, but at scale.
5
SPEAKER_01
So strong drive to simplify.
5
SPEAKER_01
From your perspective, I mean, that also probably applies to just designing systems in machine learning and otherwise, like simplify, simplify.
5
SPEAKER_01
What do you think is the secret to maintaining the startup culture in a company that grows?
5
SPEAKER_01
Is there, can you introspect that?
5
SPEAKER_00
I do think he needs someone in a powerful position with a big hammer, like Elon, who s like the cheerleader for that idea and ruthlessly pursues it.
5
SPEAKER_00
If no one has a big enough hammer, everything turns into committees, democracy within the company, process, talking to stakeholders, decision making, just everything just crumbles.
5
SPEAKER_00
If you have a big person who is also really smart and has a big hammer, things move quickly.
5
SPEAKER_01
So you said your favorite scene in Interstellar is the intense docking scene with the AI and Cooper talking, saying, Cooper, what are you doing?
5
SPEAKER_01
Docking, it s not possible.
5
SPEAKER_01
No, it s necessary.
5
SPEAKER_01
Such a good line.
5
SPEAKER_01
By the way, just so many questions there.
5
SPEAKER_01
Why an AI in that scene presumably is supposed to be able to compute a lot more than the human?
5
SPEAKER_01
It s saying it s not optimal.
5
SPEAKER_01
Why the human?
5
SPEAKER_01
I mean, that s a movie.
5
SPEAKER_01
Shouldn t the AI know much better than the human?
5
SPEAKER_01
Anyway, what do you think is the value of setting seemingly impossible goals?
5
SPEAKER_01
So like, our initial intuition, which seems like something that
5
SPEAKER_01
you have taken on that Elon espouses that where the initial intuition of the community might say this is very difficult and then you take it on anyway with a crazy deadline.
5
SPEAKER_01
You just from a human engineering perspective, have you seen the value of that?
5
SPEAKER_00
I wouldn t say that setting impossible goals exactly is a good idea, but I think setting very ambitious goals is a good idea.
5
SPEAKER_00
I think there s what I call sublinear scaling of difficulty, which means that 10x problems are not 10x hard.
5
SPEAKER_00
Usually 10x harder problem is like two or three x harder to execute on.
5
SPEAKER_00
Because if you want to improve a system by 10%, it costs some amount of work.
5
SPEAKER_00
And if you want to 10x improve the system, it doesn t cost
5
SPEAKER_00
you know, 100x amount of the work.
5
SPEAKER_00
And it s because you fundamentally change the approach.
5
SPEAKER_00
And if you start with that constraint, then some approaches are obviously dumb and not going to work.
5
SPEAKER_00
And it forces you to reevaluate.
5
SPEAKER_00
And I think it s a very interesting way of approaching problem solving.
5
SPEAKER_01
But it requires a weird kind of thinking.
5
SPEAKER_01
It s just going back to your, like, PhD days.
5
SPEAKER_01
It s like,
5
SPEAKER_01
how do you think which ideas in the machine learning community are solvable?
5
SPEAKER_01
It requires, what is that?
5
SPEAKER_01
I mean, there s the cliche of first principles thinking, but it requires to basically ignore what the community is saying, because doesn t a community in science usually draw lines of what is and isn t possible?
5
SPEAKER_01
And it s very hard to break out of that without going crazy.
5
SPEAKER_00
Yeah.
5
SPEAKER_00
I mean, I think a good example here is the deep learning revolution in some sense, because you could be in computer vision at that time during the deep learning revolution of 2012 and so on.
5
SPEAKER_00
You could be improving your computer vision stack by 10%, or we can just be saying, actually, all of this is useless.
5
SPEAKER_00
And how do I do 10x better computer vision?
5
SPEAKER_00
Well, it s not probably by tuning a hog feature detector.
5
SPEAKER_00
I need a different approach.
5
SPEAKER_00
I need something that is scalable.
5
SPEAKER_00
Going back to Richard Sutton s understanding of the philosophy of the bitter lesson, and then being like, actually, I need a much more scalable system like a neural network that in principle works, and then having some deep believers that can actually execute
6
SPEAKER_00
on that mission and make it work.
6
SPEAKER_00
So that s the 10x solution.
6
SPEAKER_01
What do you think is the timeline to solve the problem of autonomous driving?
6
SPEAKER_01
That s still in part an open question.
6
SPEAKER_00
Yeah, I think the tough thing with timelines of self-driving, obviously, is that no one has created self-driving.
6
SPEAKER_00
So it s not like, what do you think is the timeline to build this bridge?
6
SPEAKER_00
Well, we ve built million bridges before.
6
SPEAKER_00
Here s how long that takes.
6
SPEAKER_00
No one has built autonomy.
6
SPEAKER_00
It s not obvious.
6
SPEAKER_00
Some parts turn out to be much easier than others.
6
SPEAKER_00
So it s really hard to forecast.
6
SPEAKER_00
You do your best based on trend lines and so on and based on intuition.
6
SPEAKER_00
But that s why, fundamentally, it s just really hard to forecast this.
6
SPEAKER_01
So even still, being inside of it is hard to do.
6
SPEAKER_00
Yes, some things turn out to be much harder and some things turn out to be much easier.
6
SPEAKER_01
Do you try to avoid making forecasts?
6
SPEAKER_01
Because Elon doesn t avoid them, right?
6
SPEAKER_01
And heads of car companies in the past have not avoided it either.
6
SPEAKER_01
Ford and other places have made predictions that we re gonna solve level four driving by 2020, 2021, whatever.
6
SPEAKER_01
And now they re all kind of backtracking that prediction.
6
SPEAKER_01
As an AI person,
6
SPEAKER_01
Do you, for yourself, privately make predictions, or do they get in the way of your actual ability to think about a thing?
6
SPEAKER_00
Yeah, I would say what s easy to say is that this problem is tractable, and that s an easy prediction to make.
6
SPEAKER_00
It s tractable, it s going to work.
6
SPEAKER_00
Yes, it s just really hard.
6
SPEAKER_00
Some things turn out to be harder, and some things turn out to be easier.
6
SPEAKER_00
But it definitely feels tractable, and it feels like at least the team at Tesla, which is what I saw internally, is definitely on track to that.
6
SPEAKER_01
how do you form a strong representation that allows you to make a prediction about tractability?
6
SPEAKER_01
So like you re the leader of a lot of humans, you have to kind of say this is actually possible.
6
SPEAKER_01
How do you build up that intuition?
6
SPEAKER_01
It doesn t have to be even driving, it could be other tasks.
6
SPEAKER_01
What difficult tasks did you work on in your life?
6
SPEAKER_01
I mean, classification, achieving certain, just on ImageNet, certain level of superhuman level performance.
6
SPEAKER_00
Yeah, expert intuition.
6
SPEAKER_01
It s just intuition, it s belief.
6
SPEAKER_01
So just like thinking about it long enough, like studying, looking at sample data, like you said, driving.
6
SPEAKER_01
My intuition is really flawed on this.
6
SPEAKER_01
Like I don t have a good intuition about tractability.
6
SPEAKER_01
It could be anything.
6
SPEAKER_01
It could be solvable.
6
SPEAKER_01
Like the driving task could be simplified into something quite trivial, like the solution to the problem would be quite trivial.
6
SPEAKER_01
And at scale, more and more cars driving perfectly.
6
SPEAKER_01
might make the problem much easier.
6
SPEAKER_01
The more cars you have driving, like people learn how to drive correctly, not correctly, but in a way that s more optimal for a heterogeneous system of autonomous and semi-autonomous and manually driven cars, that could change stuff.
6
SPEAKER_01
Then again, also I ve spent a ridiculous number of hours just staring at pedestrians crossing streets, thinking about humans, and it feels like the way we use our eye contact
6
SPEAKER_01
it sends really strong signals and there s certain quirks and edge cases of behavior.
6
SPEAKER_01
And of course, a lot of the fatalities that happen have to do with drunk driving and both on the pedestrian side and the driver s side.
6
SPEAKER_01
So there s that problem of driving at night and all that kind of.
6
SPEAKER_01
So I wonder, you know, it s like the space,
6
SPEAKER_01
of possible solutions to autonomous driving includes so many human factor issues that it s almost impossible to predict.
6
SPEAKER_01
There could be super clean, nice solutions.
6
SPEAKER_00
Yeah.
6
SPEAKER_00
I would say definitely like to use a game analogy, there s some fog of war, but you definitely also see the frontier of improvement and you can measure historically how much you ve made progress.
6
SPEAKER_00
And I think, for example, at least what I ve seen in roughly five years at Tesla, when I joined, it barely kept lane on the highway.
6
SPEAKER_00
I think going up from Palo Alto to SF was like three or four interventions.
6
SPEAKER_00
Anytime the road would do anything geometrically or turn too much, it would just like not work.
6
SPEAKER_00
And so going from that to like a pretty competent system in five years and seeing what happens also under the hood and what the scale of which the team is operating now with respect to data and compute and everything else is just a massive progress.
6
SPEAKER_01
So you re climbing a mountain, and it s fog, but you re making a lot of progress.
6
SPEAKER_00
You re making progress, and you see what the next directions are, and you re looking at some of the remaining challenges, and they re not perturbing you, and they re not changing your philosophy, and you re not contorting yourself.
6
SPEAKER_00
You re like, actually, these are the things that we still need to do.
6
SPEAKER_01
Yeah, the fundamental components of solving the problem seem to be there, from the data engine, to the compute, to the compute on the car, to the compute for the training, all that kind of stuff.
6
SPEAKER_01
So you ve done,
6
SPEAKER_01
Over the years you ve been at Tesla, you ve done a lot of amazing breakthrough ideas in engineering, all of it, from the data engine to the human side, all of it.
6
SPEAKER_01
Can you speak to why you chose to leave Tesla?
6
SPEAKER_00
Basically, as I described, I ran, I think over time during those five years, I ve kind of gotten myself into a little bit of a managerial position.
6
SPEAKER_00
Most of my days were, you know, meetings and growing the organization and making decisions about sort of high-level strategic decisions about the team and what it should be working on and so on.
6
SPEAKER_00
It s kind of like a corporate executive role.
6
SPEAKER_00
And I can do it.
6
SPEAKER_00
I think I m OK at it.
6
SPEAKER_00
But it s not fundamentally what I enjoy.
6
SPEAKER_00
And so I think when I joined, there was no computer vision team, because Tesla was just going from the transition of using Mobileye, a third-party vendor for all of its computer vision, to having to build its computer vision system.
6
SPEAKER_00
So when I showed up, there were two people training deep neural networks.
6
SPEAKER_00
And they were training them at a computer at their legs, like down at a workstation.
6
SPEAKER_01
They re doing some kind of basic classification task.
6
SPEAKER_00
Yeah.
6
SPEAKER_00
I kind of like grew that into what I think is a fairly respectable deep learning team, a massive compute cluster, a very good data annotation organization.
6
SPEAKER_00
And I was very happy with where that was.
6
SPEAKER_00
It became quite autonomous.
6
SPEAKER_00
And so I kind of stepped away and I, you know, I m very excited to do much more technical things again.
6
SPEAKER_00
Yeah, and kind of like we focus on AGI.
6
SPEAKER_01
What was this soul-searching like?
6
SPEAKER_01
Because you took a little time off.
6
SPEAKER_01
How many mushrooms did you take?
6
SPEAKER_01
No, I m just kidding.
6
SPEAKER_01
I mean, what was going through your mind?
6
SPEAKER_01
The human lifetime is finite.
6
SPEAKER_00
Yeah.
6
SPEAKER_01
You did a few incredible things.
6
SPEAKER_01
You re one of the best teachers of AI in the world.
6
SPEAKER_01
You re one of the best, and I don t mean that, I mean that in the best possible way, you re one of the best
6
SPEAKER_01
tinkerers in the AI world, meaning like understanding the fundamentals of how something works by building it from scratch and playing with the basic intuitions.
6
SPEAKER_01
It s like Einstein, Feynman, we re all really good at this kind of stuff.
6
SPEAKER_01
Like small example of a thing to play with it, to try to understand it.
6
SPEAKER_01
So that, and obviously now with Tesla, you help build a team of machine learning
6
SPEAKER_01
um, uh, like engineers and assistant that actually accomplishes something in the real world.
6
SPEAKER_01
So given all that, like what was the soul searching like?
6
SPEAKER_00
Well, it was hard because obviously I love the company a lot and I love, I love Elon.
6
SPEAKER_00
I love Tesla.
6
SPEAKER_00
I want, um, it was always hard to leave.
6
SPEAKER_00
I love the team basically.
6
SPEAKER_00
Um, uh, but
6
SPEAKER_00
I think, actually, I would be potentially interested in revisiting it, maybe coming back at some point, working in Optimus, working in AGI at Tesla.
6
SPEAKER_00
I think Tesla is going to do incredible things.
6
SPEAKER_00
It s basically a massive, large-scale robotics company with a ton of in-house talent for doing really incredible things.
6
SPEAKER_00
And I think
6
SPEAKER_00
robots are going to be amazing.
6
SPEAKER_00
I think autonomous transportation is going to be amazing.
6
SPEAKER_00
All this is happening at Tesla.
6
SPEAKER_00
So I think it s just a really amazing organization.
6
SPEAKER_00
So being part of it and helping it along, I think was very, basically, I enjoyed that a lot.
6
SPEAKER_00
Yeah, it was basically difficult for those reasons because I love the company.
6
SPEAKER_00
But, you know, I m happy to potentially at some point come back for Act 2.
6
SPEAKER_00
But I felt like at this stage,
6
SPEAKER_00
I built the team, it felt autonomous, and I became a manager, and I wanted to do a lot more technical stuff, I wanted to learn stuff, I wanted to teach stuff, and I just kind of felt like it was a good time for a change of pace a little bit.
6
SPEAKER_01
What do you think is the best movie sequel of all time, speaking of part two?
6
SPEAKER_01
Because most of them suck.
6
SPEAKER_01
Movie sequels?
6
SPEAKER_01
Movie sequels, yeah.
6
SPEAKER_01
And you tweet about movies, so this is a tiny tangent.
6
SPEAKER_01
What s like a favorite movie sequel?
6
SPEAKER_01
Godfather Part II.
6
SPEAKER_01
Are you a fan of Godfather?
6
SPEAKER_01
Because you didn t even tweet or mention the Godfather.
6
SPEAKER_00
Yeah, I don t love that movie.
6
SPEAKER_00
I know it has a huge follow-up.
6
SPEAKER_01
We re going to edit that out.
6
SPEAKER_00
We re going to edit out the hate towards the Godfather.
6
SPEAKER_00
How dare you disrespect... I think I will make a strong statement.
6
SPEAKER_00
I don t know why.
6
SPEAKER_00
I don t know why, but I basically don t like any movie before 1995.
6
SPEAKER_00
something like that.
6
SPEAKER_00
Didn t you mention Terminator 2?
6
SPEAKER_00
Okay.
6
SPEAKER_00
Okay.
6
SPEAKER_00
That s like, Terminator 2 was a little bit later, 1990.
6
SPEAKER_01
No, I think Terminator 2 was in the 80s.
6
SPEAKER_00
And I like Terminator 1 as well.
6
SPEAKER_00
So okay, so like few exceptions, but by and large, for some reason, I don t like movies before 1995 or something.
6
SPEAKER_00
They feel very slow, the camera is like zoomed out, it s boring, it s kind of naive, it s kind of weird.
6
SPEAKER_01
And also, Terminator was very much ahead of its time.
6
SPEAKER_00
Yes, and The Godfather, there s like no AGI, so.
6
SPEAKER_01
I mean, but you have Good Will Hunting was one of the movies you mentioned, and that doesn t have any AGI either.
6
SPEAKER_01
I guess that s mathematics.
6
SPEAKER_00
Yeah, I guess occasionally I do enjoy movies that don t feature.
6
SPEAKER_01
Or like Anchorman, that has no, that s- Anchorman is so good.
6
SPEAKER_01
I don t understand, speaking of AGI, because I don t understand why Will Ferrell is so funny.
6
SPEAKER_01
It doesn t make sense, it doesn t compute, there s just something about him.
6
SPEAKER_01
And he s a singular human, because you don t get that many comedies
6
SPEAKER_01
these days, and I wonder if it has to do about the culture, or like the machine of Hollywood, or does it have to do with just we got lucky with certain people in comedy that came together, because he is a singular human.
6
SPEAKER_01
That was a ridiculous tangent, I apologize.
6
SPEAKER_01
But you mentioned humanoid robots, so what do you think about Optimus, about TeslaBot?
6
SPEAKER_01
Do you think we ll have robots in the factory and in the home in 10, 20, 30, 40, 50 years?
6
SPEAKER_00
Yeah, I think it s a very hard project.
6
SPEAKER_00
I think it s going to take a while,
6
SPEAKER_00
Who else is going to build human robots at scale?
6
SPEAKER_00
And I think it is a very good form factor to go after, because like I mentioned, the world is designed for human form factor.
6
SPEAKER_00
These things would be able to operate our machines.
6
SPEAKER_00
They would be able to sit down in chairs, potentially even drive cars.
6
SPEAKER_00
Basically, the world is designed for humans.
6
SPEAKER_00
That s the form factor you want to invest into and make work over time.
6
SPEAKER_00
I think there s another school of thought, which is, okay, pick a problem and design a robot to it.
6
SPEAKER_00
But actually designing a robot and getting a whole data engine and everything behind it to work is actually an incredibly hard problem.
6
SPEAKER_00
So it makes sense to go after general interfaces that, okay, they are not perfect for any one given task, but they actually have the generality of just with a prompt with English able to do something across.
6
SPEAKER_00
And so I think it makes a lot of sense to go after a general interface in the physical world.
6
SPEAKER_00
And I think it s a very difficult project.
6
SPEAKER_00
I think it s going to take time.
6
SPEAKER_00
But I ve seen no other company that can execute on that vision.
6
SPEAKER_00
I think it s going to be amazing.
6
SPEAKER_00
Basically, physical labor.
6
SPEAKER_00
If you think transportation is a large market, try physical labor.
6
SPEAKER_00
It s insane.
6
SPEAKER_01
But it s not just physical labor.
6
SPEAKER_01
To me, the thing that s also exciting is social robotics.
6
SPEAKER_01
So the relationship we ll have on different levels with those robots.
6
SPEAKER_01
That s why I was really excited to see Optimus.
6
SPEAKER_01
People have criticized me for the excitement.
6
SPEAKER_01
But I ve worked with a lot of research labs that do humanoid-legged robots, Boston Dynamics, Unitree.
6
SPEAKER_01
There s a lot of companies that do legged robots.
6
SPEAKER_01
but that s the elegance of the movement is a tiny, tiny part of the big picture.
6
SPEAKER_01
So integrating, the two big exciting things to me about Tesla doing humanoid or any legged robots is,
6
SPEAKER_01
clearly integrating it into the data engine.
6
SPEAKER_01
So the data engine aspect, so the actual intelligence for the perception and the control and the planning and all that kind of stuff, integrating into the fleet that you mentioned, right?
6
SPEAKER_01
And then, speaking of fleet, the second thing is the mass manufacturers, just knowing,
6
SPEAKER_01
culturally driving towards a simple robot that s cheap to produce at scale.
6
SPEAKER_01
And doing that well, having experience to do that well, that changes everything.
6
SPEAKER_01
That s a very different culture and style than Boston Dynamics, who, by the way, those robots,
6
SPEAKER_01
are just, the way they move, it s like, it ll be a very long time before Tesla can achieve the smoothness of movement.
6
SPEAKER_01
But that s not what it s about.
6
SPEAKER_01
It s about the entirety of the system, like we talked about the data engine and the fleet.
6
SPEAKER_01
And that s super exciting, even the initial sort of models.
6
SPEAKER_01
But that too was really surprising, that in a few months you can get a prototype.
6
SPEAKER_00
And the reason that happened very quickly is, as you alluded to, there s a ton of copy-paste from what s happening on the autopilot.
6
SPEAKER_00
A lot.
6
SPEAKER_00
The amount of expertise that came out of the woodworks at Tesla for building the human robot was incredible to see.
6
SPEAKER_00
Basically, Elon said, at one point, we re doing this.
6
SPEAKER_00
And then next day, basically, all these CAD models started to appear.
6
SPEAKER_00
And people talking about the supply chain and manufacturing.
6
SPEAKER_00
And people showed up with screwdrivers and everything the other day and started to put together the body.
6
SPEAKER_00
And I was like, whoa.
6
SPEAKER_00
All these people exist at Tesla.
6
SPEAKER_00
And fundamentally, building a car is actually not that different from building a robot.
6
SPEAKER_00
And that is true, not just for the hardware pieces.
6
SPEAKER_00
And also, let s not forget hardware, not just for a demo,
6
SPEAKER_00
manufacturing of that hardware at scale is like a whole different thing.
6
SPEAKER_00
But for software as well, basically this robot currently thinks it s a car.
6
SPEAKER_01
It s going to have a midlife crisis at some point.
6
SPEAKER_00
It thinks it s a car.
6
SPEAKER_00
Some of the earlier demos, actually, we were talking about potentially doing them outside in the parking lot, because that s where all of the computer vision was working out of the box instead of inside.
6
SPEAKER_00
But all the operating system, everything just copy-pastes.
6
SPEAKER_00
Computer vision, mostly copy-pastes.
6
SPEAKER_00
I mean, you have to retrain the neural nets, but the approach and everything, and data engine, and offline trackers, and the way we go about the occupancy tracker, and so on, everything copy-pastes.
6
SPEAKER_00
You just need to retrain the neural nets.
6
SPEAKER_00
And then the planning control, of course, has to change quite a bit.
6
SPEAKER_00
But there s a ton of copy-paste from what s happening at Tesla.
6
SPEAKER_00
And so if you were to go with the goal of like, OK, let s build a million human robots, and you re not Tesla, that s a lot to ask.
6
SPEAKER_00
If you re Tesla, it s actually like, it s not that crazy.
6
SPEAKER_01
And then the follow-up question is then how difficult, just like with driving, how difficult is the manipulation task, such that it can have an impact at scale?
6
SPEAKER_01
depending on the context, the really nice thing about robotics is that, unless you do manufacturing and that kind of stuff, is there is more room for error.
6
SPEAKER_01
Driving is so safety critical, and also time critical.
6
SPEAKER_01
A robot is allowed to move slower.
6
SPEAKER_01
which is nice.
6
SPEAKER_00
Yes.
6
SPEAKER_00
I think it s going to take a long time, but the way you want to structure the development is you need to say, okay, it s going to take a long time.
6
SPEAKER_00
How can I set up the product development roadmap so that I m making revenue along the way?
6
SPEAKER_00
I m not setting myself up for a zero-one loss function where it doesn t work until it works.
6
SPEAKER_00
You don t want to be in that position.
6
SPEAKER_00
You want to make it useful almost immediately, and then you want to slowly deploy it and generalize it at scale.
6
SPEAKER_00
And you want to set up your data engine, your improvement loops, the telemetry, the evaluation, the harness, and everything.
6
SPEAKER_00
And you want to improve the product over time incrementally.
6
SPEAKER_00
And you re making revenue along the way.
6
SPEAKER_00
That s extremely important.
6
SPEAKER_00
Because otherwise, you cannot build these large undertakings just don t make sense economically.
6
SPEAKER_00
And also, from the point of view of the team working on it, they need the dopamine along the way.
6
SPEAKER_00
They re not just going to make a promise about this being useful
6
SPEAKER_00
This is going to change the world in 10 years when it works.
6
SPEAKER_00
This is not where you want to be.
6
SPEAKER_00
You want to be in a place like, I think, Autopilot is today, where it s offering increased safety and convenience of driving today.
6
SPEAKER_00
People pay for it.
6
SPEAKER_00
People like it.
6
SPEAKER_00
People will purchase it.
6
SPEAKER_00
And then you also have the greater mission that you re working towards.
6
SPEAKER_01
And you see that, so the dopamine for the team, that was a source of happiness and satisfaction.
6
SPEAKER_00
Yes, 100%.
6
SPEAKER_00
You re deploying this, people like it, people drive it, people pay for it, they care about it.
6
SPEAKER_00
There s all these YouTube videos.
6
SPEAKER_00
Your grandma drives it, she gives you feedback.
6
SPEAKER_00
People like it, people engage with it.
6
SPEAKER_00
You engage with it, huge.
6
SPEAKER_01
Do people that drive Teslas recognize you and give you love?
6
SPEAKER_01
Like, hey, thanks for this nice feature that it s doing.
6
SPEAKER_00
Yeah, I think the tricky thing is like some people really love you.
6
SPEAKER_00
Some people, unfortunately, like you re working on something that you think is extremely valuable, useful, etc.
6
SPEAKER_00
Some people do hate you.
7
SPEAKER_00
There s a lot of people who hate me and the team and the whole project.
7
SPEAKER_01
Are they Tesla drivers?
7
SPEAKER_00
In many cases, they re not, actually.
7
SPEAKER_01
Yeah, that actually makes me sad about humans, or the current ways that humans interact.
7
SPEAKER_01
I think that s actually fixable.
7
SPEAKER_01
I think humans want to be good to each other.
7
SPEAKER_01
I think Twitter and social media is part of the mechanism that actually somehow makes the negativity more viral, that it doesn t deserve, like disproportionately add a viral boost to the negativity.
7
SPEAKER_01
But I wish people would just get excited about
7
SPEAKER_01
So suppress some of the jealousy, some of the ego, and just get excited for others.
7
SPEAKER_01
And then there s a karma aspect to that.
7
SPEAKER_01
You get excited for others, they ll get excited for you.
7
SPEAKER_01
Same thing in academia.
7
SPEAKER_01
If you re not careful, there is like a dynamical system there.
7
SPEAKER_01
If you think of in silos and get jealous of somebody else being successful, that actually, perhaps counterintuitively, leads to less productivity of you as a community and you individually.
7
SPEAKER_01
I feel like if you keep celebrating others,
7
SPEAKER_01
that actually makes you more successful.
7
SPEAKER_01
And I think people haven t, depending on the industry, haven t quite learned that yet.
7
SPEAKER_00
Some people are also very negative and very vocal, so they re very prominently featured.
7
SPEAKER_00
But actually, there s a ton of people who are cheerleaders, but they re silent cheerleaders.
7
SPEAKER_00
And when you talk to people just in the world, they will tell you, oh, it s amazing, it s great.
7
SPEAKER_00
Especially people who understand how difficult it is to get this stuff working.
7
SPEAKER_00
People who have built products, makers, entrepreneurs, making this work and changing something,
7
SPEAKER_00
is incredibly hard.
7
SPEAKER_00
Those people are more likely to cheerlead you.
7
SPEAKER_01
Well, one of the things that makes me sad is some folks in the robotics community don t do the cheerleading and they should.
7
SPEAKER_01
There s a, cause they know how difficult it is.
7
SPEAKER_01
Well, they actually sometimes don t know how difficult it is to create a product that scale, right?
7
SPEAKER_01
They actually deploy it in the real world.
7
SPEAKER_01
A lot of the development of robots and AI system is done on very specific, small benchmarks.
7
SPEAKER_00
And as opposed to real world conditions.
7
SPEAKER_00
Yeah, I think it s really hard to work on robotics in an academic setting.
7
SPEAKER_01
Or AI systems that apply in the real world.
7
SPEAKER_01
You ve criticized, you flourished and loved for a time the ImageNet, the famed ImageNet dataset, and have recently had some words of criticism that the academic research ML community gives a little too much love still to the ImageNet, or like those kinds of benchmarks.
7
SPEAKER_01
Can you speak to the strengths and weaknesses of datasets
7
SPEAKER_01
used in machine learning research?
7
SPEAKER_00
Actually, I don t know that I recall a specific instance where I was unhappy or criticizing ImageNet.
7
SPEAKER_00
I think ImageNet has been extremely valuable.
7
SPEAKER_00
It was basically a benchmark that allowed the deep learning community to demonstrate that deep neural networks actually work.
7
SPEAKER_00
there s a massive value in that.
7
SPEAKER_00
So I think ImageNet was useful, but basically it s become a bit of an MNIST at this point.
7
SPEAKER_00
So MNIST is like little two 28 by 28 grayscale digits.
7
SPEAKER_00
There s kind of a joke data set that everyone like just crushes.
7
SPEAKER_01
There s still papers written on MNIST though, right?
7
SPEAKER_00
Maybe they shouldn t.
7
SPEAKER_01
Like strong papers, like papers that focus on like, how do we learn with a small amount of data, that kind of stuff.
7
SPEAKER_00
Yeah, I could see that being helpful, but not in sort of like mainline computer vision research anymore, of course.
7
SPEAKER_01
I think the way I ve heard you somewhere, maybe I m just imagining things, but I think you said like ImageNet was a huge contribution to the community for a long time, and now it s time to move past those kinds of... Well, ImageNet has been crushed.
7
SPEAKER_00
I mean, you know, the error rates are...
7
SPEAKER_00
Yeah, we re getting like 90% accuracy in 1000 classification way prediction.
7
SPEAKER_00
And I ve seen those images and it s like really high.
7
SPEAKER_00
That s really, that s really good.
7
SPEAKER_00
If I remember correctly, the top five error rate is now like 1% or something.
7
SPEAKER_01
Given your experience with a gigantic real world dataset, would you like to see benchmarks move in a certain directions that the research community uses?
7
SPEAKER_00
Unfortunately, I don t think academics currently have the next ImageNet.
7
SPEAKER_00
We ve obviously, I think we ve crushed MNIST.
7
SPEAKER_00
We ve basically kind of crushed ImageNet.
7
SPEAKER_00
And there s no next sort of big benchmark that the entire community rallies behind and uses.
7
SPEAKER_00
you know, for further development of these networks.
7
SPEAKER_01
Yeah, I wonder what it takes for a dataset to captivate the imagination of everybody, like where they all get behind it.
7
SPEAKER_01
That could also need like a leader, right?
7
SPEAKER_01
Somebody with popularity.
7
SPEAKER_01
I mean, yeah, why did ImageNet take off?
7
SPEAKER_01
Or is it just the accident of history?
7
SPEAKER_00
It was the right amount of difficult.
7
SPEAKER_00
It was the right amount of difficult and simple and interesting enough.
7
SPEAKER_00
It just kind of like, it was the right time for that kind of a dataset.
7
SPEAKER_01
Question from Reddit.
7
SPEAKER_01
What are your thoughts on the role that synthetic data and game engines will play in the future of neural net model development?
7
SPEAKER_00
I think as neural nets converge to humans, the value of simulation to neural nets will be similar to value of simulation to humans.
7
SPEAKER_00
So people use simulation because they can learn something in that kind of a system.
7
SPEAKER_00
and without having to actually experience it.
7
SPEAKER_01
But are you referring to the simulation we do in our head?
7
SPEAKER_00
No, sorry, simulation, I mean like video games or other forms of simulation for various professionals.
7
SPEAKER_01
So let me push back on that, because maybe there s simulation that we do in our heads, like simulate, if I do this, what do I think will happen?
7
SPEAKER_00
Okay.
7
SPEAKER_00
That s like internal simulation.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
Internal.
7
SPEAKER_01
Isn t that what we re doing?
7
SPEAKER_01
Assuming before we act?
7
SPEAKER_00
Oh yeah, but that s independent from like the use of simulation in the sense of like computer games or using simulation for training set creation or... Is it independent or is it just loosely correlated?
7
SPEAKER_01
Because like, isn t that useful to do like counterfactual or like edge case simulation to like
7
SPEAKER_01
you know, what happens if there s a nuclear war?
7
SPEAKER_01
What happens if there s, you know, like those kinds of things.
7
SPEAKER_00
Yeah, that s a different simulation from like Unreal Engine.
7
SPEAKER_00
That s how I interpreted the question.
7
SPEAKER_01
Ah, so like simulation of the average case.
7
SPEAKER_01
What s Unreal Engine?
7
SPEAKER_01
What do you mean by Unreal Engine?
7
SPEAKER_01
So simulating a world, the physics of that world,
7
SPEAKER_01
Why is that different?
7
SPEAKER_01
Because you also can add behavior to that world, and you can try all kinds of stuff, right?
7
SPEAKER_01
You could throw all kinds of weird things into it.
7
SPEAKER_01
A real engine is not just about, I mean, I guess it is about simulating the physics of the world, it s also doing something with that.
7
SPEAKER_00
Yeah, the graphics, the physics, and the agents that you put into the environment and stuff like that.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
See, I feel like you said that it s not that important, I guess, for the future of AI development.
7
SPEAKER_01
Is that correct to interpret it that way?
7
SPEAKER_00
Well, I think humans use simulators and they find them useful.
7
SPEAKER_00
And so computers will use simulators and find them useful.
7
SPEAKER_01
Okay, so you re saying it s not, I don t use simulators very often.
7
SPEAKER_01
I play a video game every once in a while, but I don t think I derive any wisdom about my own existence from those video games.
7
SPEAKER_01
It s a momentary escape from reality versus a source of wisdom about reality.
7
SPEAKER_01
So I think that s a very polite way of saying simulation is not that useful.
7
SPEAKER_00
Yeah, maybe not.
7
SPEAKER_00
I don t see it as a fundamental, really important part of training neural nets currently.
7
SPEAKER_00
But I think as neural nets become more and more powerful, I think you will need fewer examples to train additional behaviors.
7
SPEAKER_00
And simulation is, of course, there s a domain gap in a simulation.
7
SPEAKER_00
That s not the real world.
7
SPEAKER_00
That s slightly something different.
7
SPEAKER_00
but with a powerful enough neural net, you need, the domain gap can be bigger, I think, because neural net will sort of understand that even though it s not the real world, it has all this high-level structure that I m supposed to be able to learn from.
7
SPEAKER_01
So the neural net will actually, yeah, it will be able to leverage,
7
SPEAKER_01
the synthetic data better by closing the gap, but understanding in which ways this is not real data.
7
SPEAKER_01
I m ready to do better questions next time.
7
SPEAKER_01
That was a question, I m just kidding.
7
SPEAKER_01
All right, so is it possible, do you think, speaking of MNIST, to construct neural nets and training processes that require very little data?
7
SPEAKER_01
So we ve been talking about huge data sets like the internet for training.
7
SPEAKER_01
I mean, one way to say that is, like you said, like the querying itself is another level of training, I guess, and that requires a little data.
7
SPEAKER_01
But do you see any value in doing research
7
SPEAKER_01
and kind of going down the direction of, can we use very little data to train, to construct a knowledge base?
7
100%.
7
SPEAKER_00
I just think like at some point you need a massive data set.
7
SPEAKER_00
And then when you pre-train your massive neural net and get something that, you know, is like a GPT or something, then you re able to be very efficient at training any arbitrary new task.
7
SPEAKER_00
So a lot of these GPTs, you can do tasks like sentiment analysis or translation or so on just by being prompted with very few examples.
7
SPEAKER_00
Here s the kind of thing I want you to do.
7
SPEAKER_00
Here s an input sentence.
7
SPEAKER_00
Here s the translation into German.
7
SPEAKER_00
Input sentence, translation to German.
7
SPEAKER_00
Input sentence, blank, and the neural net will complete the translation to German just by looking at the example you ve provided.
7
SPEAKER_00
And so that s an example of a very few-shot learning in the activations of the neural net instead of the weights of the neural net.
7
SPEAKER_00
And so I think
7
SPEAKER_00
Basically, just like humans, neural nets will become very data efficient at learning any other new task.
7
SPEAKER_00
But at some point, you need a massive data set to pre-train your network.
7
SPEAKER_01
To get that, and probably we humans have something like that.
7
SPEAKER_01
Do we have something like that?
7
SPEAKER_01
Do we have a passive, in the background, background model constructing thing that just runs all the time in a self-supervised way?
7
SPEAKER_01
We re not conscious of it.
7
SPEAKER_00
I think humans definitely, I mean, obviously we learn a lot during our lifespan, but also we have a ton of hardware that helps us at initialization coming from evolution.
7
SPEAKER_00
And so I think that s also a really big component.
7
SPEAKER_00
A lot of people in the field, I think they just talk about the amounts of seconds that a person has lived pretending that this is a tabula rasa, sort of like a zero initialization of a neural net.
7
SPEAKER_00
And it s not.
7
SPEAKER_00
You can look at a lot of animals, like, for example, zebras.
7
SPEAKER_00
Zebras get born, and they see, and they can run.
7
SPEAKER_00
There s zero training data in their lifespan.
7
SPEAKER_00
They can just do that.
7
SPEAKER_00
So somehow, I have no idea how evolutionists found a way to encode these algorithms and these neural net initializations that are extremely good into ATCGs.
7
SPEAKER_00
And I have no idea how this works, but apparently it s possible, because here s a proof by existence.
7
SPEAKER_01
There s something magical about going from a single cell to an organism that is born to the first few years of life.
7
SPEAKER_01
I kind of like the idea that the reason we don t remember anything about the first few years of our life is that it s a really painful process.
7
SPEAKER_01
Like, it s a very difficult, challenging training process.
7
SPEAKER_01
Like, intellectually.
7
SPEAKER_01
Like...
7
SPEAKER_01
And maybe, yeah, I mean, I don t, why don t we remember any of that?
7
SPEAKER_01
There might be some crazy training going on and that, maybe that s the background model training that is very painful.
7
SPEAKER_01
And so it s best for the system once it s trained not to remember how it s constructed.
7
SPEAKER_00
I think it s just like the hardware for long-term memory is just not fully developed.
7
SPEAKER_00
Sure.
7
SPEAKER_00
I kind of feel like the first few years of,
7
SPEAKER_00
of infants is not actually like learning, it s brain maturing.
7
SPEAKER_00
We re born premature.
7
SPEAKER_00
There s a theory along those lines because of the birth canal and the swelling of the brain.
7
SPEAKER_00
And so we re born premature and then the first few years we re just, the brain s maturing.
7
SPEAKER_00
And then there s some learning eventually.
7
SPEAKER_00
That s my current view on it.
7
SPEAKER_01
What do you think, do you think neural nets can have long-term memory?
7
SPEAKER_01
Like that approach is something like humans.
7
SPEAKER_01
Do you think there needs to be another meta architecture on top of it to add something like a knowledge base that learns facts about the world and all that kind of stuff?
7
SPEAKER_00
Yes, but I don t know to what extent it will be explicitly constructed.
7
SPEAKER_00
It might take unintuitive forms where you are telling the GPT, like, hey, you have a declarative memory bank to which you can store and retrieve data from.
7
SPEAKER_00
And whenever you encounter some information that you find useful, just save it to your memory bank.
7
SPEAKER_00
And here s an example of something you have retrieved, and here s how you save it, and here s how you load from it.
7
SPEAKER_00
You just say, load.
7
SPEAKER_00
whatever, you teach it in text, in English, and then it might learn to use a memory bank from that.
7
SPEAKER_01
Oh, so the neural net is the architecture for the background model, the base thing, and then everything else is just on top of it.
7
SPEAKER_00
That s pretty easy too.
7
SPEAKER_00
It s not just text, right?
7
SPEAKER_00
You re giving it gadgets and gizmos.
7
SPEAKER_00
So you re teaching some kind of a special language by which it can save arbitrary information and retrieve it at a later time.
7
SPEAKER_00
And you re telling it about these special tokens and how to arrange them to use these interfaces.
7
SPEAKER_00
It s like, hey, you can use a calculator.
7
SPEAKER_00
Here s how you use it.
7
SPEAKER_00
Just do five, three plus four, one equals.
7
SPEAKER_00
And when equals is there, a calculator will actually read out the answer and you don t have to calculate it yourself.
7
SPEAKER_00
And you just like tell it in English, this might actually work.
7
SPEAKER_01
Do you think in that sense, Gato is interesting, the deep mind system that it s not just a language, but actually throws it all in the same pile, images, actions, all that kind of stuff.
7
SPEAKER_01
That s basically what we re moving towards.
7
SPEAKER_00
Yeah, I think so.
7
SPEAKER_00
So Gado is very much a kitchen sink approach to like reinforcement learning, lots of different environments with a single fixed transformer model, right?
7
SPEAKER_00
I think it s a very sort of early result in that realm, but I think, yeah, it s along the lines of what I think things will eventually look like.
7
SPEAKER_01
Right.
7
SPEAKER_01
So this is the early days of a system that eventually will look like this, like from a rich sudden perspective.
7
SPEAKER_00
Yeah, I m not super huge fan of, I think, all these interfaces that look very different.
7
SPEAKER_00
I would want everything to be normalized into the same API.
7
SPEAKER_00
So, for example, ScreenPixels, very same API.
7
SPEAKER_00
Instead of having different world environments that have very different physics and joint configurations and appearances and whatever, and you re having some kind of special tokens for different games that you can plug, I d rather just normalize everything to a single interface.
7
SPEAKER_00
So it looks the same to the neural net, if that makes sense.
7
SPEAKER_00
So it s all going to be pixel-based pong in the end?
7
SPEAKER_00
I think so.
7
SPEAKER_01
Okay.
7
SPEAKER_01
Let me ask you about your own personal life.
7
SPEAKER_01
A lot of people want to know, you re one of the most productive and brilliant people in the history of AI.
7
SPEAKER_01
What does a productive day in the life of Andrej Karpathy look like?
7
SPEAKER_01
What time do you wake up?
7
SPEAKER_01
Because imagine some kind of dance between the average productive day and a perfect productive day.
7
SPEAKER_01
So the perfect productive day is the thing we strive towards, and the average is kind of what it kind of converges to, given all the mistakes and human eventualities and so on.
7
SPEAKER_01
So what time do you wake up?
7
SPEAKER_01
Are you a morning person?
7
SPEAKER_00
I m not a morning person.
7
SPEAKER_00
I m a night owl, for sure.
7
SPEAKER_01
Is it stable or not?
7
SPEAKER_00
It s semi-stable, like eight or nine or something like that.
7
SPEAKER_00
During my PhD, it was even later.
7
SPEAKER_00
I used to go to sleep usually at 3 a.m.
7
SPEAKER_00
I think the a.m.
7
SPEAKER_00
hours are precious and very interesting time to work because everyone is asleep.
7
SPEAKER_00
At 8 a.m.
7
SPEAKER_00
or 7 a.m., the East Coast is awake.
7
SPEAKER_00
So there s already activity.
7
SPEAKER_00
There s already some text messages, whatever.
7
SPEAKER_00
There s stuff happening.
7
SPEAKER_00
You can go on some news website and there s stuff happening.
7
SPEAKER_00
It s distracting.
7
SPEAKER_00
At 3 a.m., everything is totally quiet.
7
SPEAKER_00
And so you re not going to be bothered and you have solid chunks of time to do work.
7
SPEAKER_00
So I like those periods, night owl by default.
7
SPEAKER_00
And then I think like productive time basically, what I like to do is you need to like build some momentum on a problem without too much distraction.
7
SPEAKER_00
And you need to load your RAM, your working memory with that problem.
7
SPEAKER_00
And then you need to be obsessed with it when you re taking shower, when you re falling asleep.
7
SPEAKER_00
You need to be obsessed with the problem and it s fully in your memory and you re ready to wake up and work on it right there.
7
SPEAKER_01
So is this in a temporal scale of a single day or a couple of days, a week, a month?
7
SPEAKER_00
So I can t talk about one day basically in isolation because it s a whole process.
7
SPEAKER_00
When I want to get productive in the problem, I feel like I need a span of a few days where I can really get in on that problem.
7
SPEAKER_00
And I don t want to be interrupted.
7
SPEAKER_00
And I m going to just be completely obsessed with that problem.
7
SPEAKER_00
And that s where I do most of my good work, I would say.
7
SPEAKER_01
You ve done a bunch of cool little projects in a very short amount of time, very quickly.
7
SPEAKER_01
So that requires you just focusing on it.
7
SPEAKER_00
Yeah, basically, I need to load my working memory with the problem, and I need to be productive.
7
SPEAKER_00
Because there s always a huge fixed cost to approaching any problem.
7
SPEAKER_00
I was struggling with this, for example, at Tesla, because I want to work on a small side project.
7
SPEAKER_00
But OK, you first need to figure out, OK, I need to SSH into my cluster.
7
SPEAKER_00
I need to bring up a VS Code editor so I can work on this.
7
SPEAKER_00
I run into some stupid error because of some reason.
7
SPEAKER_00
You re not at a point where you can be just productive right away.
7
SPEAKER_00
You are facing barriers.
7
SPEAKER_00
And so it s about really
7
SPEAKER_00
removing all that barrier and you re able to go into
8
SPEAKER_00
to the problem and you have the full problem loaded in your memory.
8
SPEAKER_01
And somehow avoiding distractions of all different forms, like news stories, emails, but also distractions from other interesting projects that you previously worked on or currently working on and so on.
8
SPEAKER_01
You just want to really focus your mind.
8
SPEAKER_00
And I mean, I can take some time off for distractions and in between, but I think it can t be too much.
8
SPEAKER_00
You know, most of your day is sort of like spent on that problem.
8
SPEAKER_00
And then, you know,
8
SPEAKER_00
drink coffee, I have my morning routine, I look at some news, Twitter, Hacker News, Wall Street Journal, et cetera.
8
SPEAKER_01
It s great.
8
SPEAKER_01
So basically, you wake up, you have some coffee, are you trying to get to work as quickly as possible, or do you take in this diet of what the hell s happening in the world first?
8
SPEAKER_00
I do find it interesting to know about the world.
8
SPEAKER_00
I don t know that it s useful or good, but it is part of my routine right now.
8
SPEAKER_00
So I do read through a bunch of news articles, and I wanna be informed.
8
SPEAKER_00
I m suspicious of it.
8
SPEAKER_00
I m suspicious of the practice, but currently that s where I am.
8
SPEAKER_01
Oh, you mean suspicious about the positive effect of that practice on your productivity and your well-being.
8
SPEAKER_01
My well-being psychologically, yeah.
8
SPEAKER_01
And also on your ability to deeply understand the world, because there s a bunch of sources of information, you re not really focused on deeply integrating it.
8
SPEAKER_00
Yeah, it s a little distracting.
8
SPEAKER_01
In terms of a perfectly productive day, for how long of a stretch of time in one session do you try to work and focus on a thing?
8
SPEAKER_01
Is it a couple hours?
8
SPEAKER_01
Is it one hour?
8
SPEAKER_01
Is it 30 minutes?
8
SPEAKER_01
Is it 10 minutes?
8
SPEAKER_00
I can probably go like a small few hours and then I need some breaks in between for like food and stuff.
8
SPEAKER_00
But I think it s still really hard to accumulate hours.
8
SPEAKER_00
I was using a tracker that told me exactly how much time I spent coding any one day.
8
SPEAKER_00
And even on a very productive day, I still spent only six or eight hours.
8
SPEAKER_00
And it s just because there s so much padding.
8
SPEAKER_00
Commute, talking to people, food, etc.
8
SPEAKER_00
There s the cost of life.
8
SPEAKER_00
Just living and sustaining and homeostasis and just maintaining yourself as a human is very high.
8
SPEAKER_01
And that there seems to be a desire within the human mind to participate in society that creates that padding.
8
SPEAKER_01
The most productive days I ve ever had is just completely from start to finish just tuning out everything and just sitting there.
8
SPEAKER_01
And then you could do more than six and eight hours.
8
SPEAKER_01
Is there some wisdom about what gives you strength to do tough days of long focus?
8
SPEAKER_00
Yeah, just like whenever I get obsessed about a problem, something just needs to work, something just needs to exist.
8
SPEAKER_01
It needs to exist, so you re able to deal with bugs and programming issues and technical issues and design decisions that turn out to be the wrong ones.
8
SPEAKER_01
You re able to think through all of that, given that you want a thing to exist.
8
SPEAKER_00
Yeah, it needs to exist.
8
SPEAKER_00
And then I think, to me, also a big factor is, you know, are other humans are going to appreciate it?
8
SPEAKER_00
Are they going to like it?
8
SPEAKER_00
That s a big part of my motivation.
8
SPEAKER_00
If I m helping humans and they seem happy, they say nice things, they tweet about it or whatever, that gives me pleasure because I m doing something useful.
8
SPEAKER_01
So, like, you do see yourself sharing it with the world, like whether it s on GitHub, whether it s a blog post or through videos.
8
SPEAKER_00
Yeah, I was thinking about it.
8
SPEAKER_00
Like, suppose I did all these things but did not share them.
8
SPEAKER_00
I don t think I would have the same amount of motivation that I can build up.
8
SPEAKER_01
You enjoy the feeling of other people gaining value and happiness from the stuff you ve created.
8
SPEAKER_01
What about diet?
8
SPEAKER_01
I saw you played with intermittent fasting.
8
SPEAKER_01
Do you fast?
8
SPEAKER_01
Does that help?
8
SPEAKER_00
I play with everything.
8
SPEAKER_01
With the things you play, what s been...
8
SPEAKER_01
most beneficial to your ability to mentally focus on a thing, and just mental productivity and happiness.
8
SPEAKER_01
You still fast?
8
SPEAKER_00
Yeah, I still fast, but I do intermittent fasting, but really what it means at the end of the day is I skip breakfast.
8
SPEAKER_00
So I do 18.6 roughly by default when I m in my steady state.
8
SPEAKER_00
If I m traveling or doing something else, I will break the rules, but in my steady state, I do 18.6, so I eat only from 12 to six.
8
SPEAKER_00
Not a hard rule, and I break it often, but that s my default.
8
SPEAKER_00
And then, yeah, I ve done a bunch of random experiments.
8
SPEAKER_00
For the most part right now, where I ve been for the last year and a half, I want to say, is I m plant-based or plant-forward.
8
SPEAKER_00
I heard plant-forward.
8
SPEAKER_00
It sounds better.
8
SPEAKER_00
I don t actually know the differences, but it sounds better in my mind.
8
SPEAKER_00
But it just means I prefer plant-based food.
8
SPEAKER_01
Raw or cooked?
8
SPEAKER_00
I prefer cooked and plant-based.
8
SPEAKER_01
So plant-based, forgive me, I don t actually know how wide the category of plant entails.
8
SPEAKER_00
Like a chickpeas?
8
SPEAKER_00
Like a chickpeas?
8
SPEAKER_00
And you can flex.
8
SPEAKER_00
And you just prefer to eat plants.
8
SPEAKER_00
And you re not making, you re not trying to influence other people.
8
SPEAKER_00
And if someone is, you come to someone s house party and they serve you a steak that they re really proud of, you will eat it.
8
SPEAKER_00
Yes.
8
SPEAKER_01
Right, so you re not judgmental.
8
SPEAKER_01
That s beautiful.
8
SPEAKER_01
I mean, that s, I m the flip side of that, but I m very sort of flexible.
8
SPEAKER_01
Have you tried doing one meal a day?
8
SPEAKER_00
I have, accidentally, not consistently.
8
SPEAKER_00
But I ve accidentally had that.
8
SPEAKER_00
I don t like it.
8
SPEAKER_00
I think it makes me feel not good.
8
SPEAKER_00
It s too much of a hit.
8
SPEAKER_00
And so currently I have about two meals a day, 12 and six.
8
SPEAKER_01
I do that nonstop.
8
SPEAKER_01
I m doing it now.
8
SPEAKER_01
I do one meal a day.
8
SPEAKER_01
It s interesting.
8
SPEAKER_01
It s an interesting feeling.
8
SPEAKER_01
Have you ever fasted longer than a day?
8
SPEAKER_00
Yeah, I ve done a bunch of water fasts because I was curious what happens.
8
SPEAKER_01
Anything interesting?
8
SPEAKER_00
Yeah, I would say so.
8
SPEAKER_00
I mean, you know, what s interesting is that you re hungry for two days and then starting day three or so, you re not hungry.
8
SPEAKER_00
It s like such a weird feeling because you haven t eaten in a few days and you re not hungry.
8
SPEAKER_01
Isn t that weird?
8
SPEAKER_01
It s really weird.
8
SPEAKER_01
One of the many weird things about human biology, it figures something out.
8
SPEAKER_01
It finds another source of energy or something like that, or relaxes the system.
8
SPEAKER_01
I don t know how it works.
8
SPEAKER_00
Yeah, the body is like, you re hungry, you re hungry, and then it just gives up.
8
SPEAKER_00
It s like, okay, I guess we re fasting now.
8
SPEAKER_00
And then it just kind of like focuses on trying to make you not hungry and, you know, not feel the damage of that and trying to give you some space to figure out the food situation.
8
SPEAKER_01
So are you still to this day most productive at night?
8
SPEAKER_00
I would say I am, but it is really hard to maintain my PhD schedule, especially when I was, say, working at Tesla and so on.
8
SPEAKER_00
It s a non-starter.
8
SPEAKER_00
So but even now, like, you know, people want to meet for
8
SPEAKER_00
various events, society lives in a certain period of time and you sort of have to work with that.
8
SPEAKER_01
It s hard to do a social thing and then after that return and do work.
8
SPEAKER_01
Yeah, it s just really hard.
8
SPEAKER_01
That s why I try when I do social things, I try not to do too much drinking so I can return and continue doing work.
8
SPEAKER_01
But at Tesla, is there a convergence, Tesla, but any company, is there a convergence towards the schedule, or is there more, is that how humans behave when they collaborate?
8
SPEAKER_01
I need to learn about this.
8
SPEAKER_01
Do they try to keep a consistent schedule where you re all awake at the same time?
8
SPEAKER_00
I mean, I do try to create a routine, and I try to create a steady state in which I m comfortable in.
8
SPEAKER_00
So I have a morning routine, I have a day routine, I try to keep things to a steady state.
8
SPEAKER_00
things are predictable and then you can sort of just like, your body just sort of like sticks to that.
8
SPEAKER_00
And if you try to stress that a little too much, it will create, you know, when you re traveling and you re dealing with jet lag, you re not able to really ascend to, you know, where you need to go.
8
SPEAKER_01
In your mind.
8
SPEAKER_01
Yeah, that s weird too about humans with the habits and stuff.
8
SPEAKER_01
What are your thoughts on work-life balance throughout a human lifetime?
8
SPEAKER_01
So Tesla, in part, was known for pushing people to their limits in terms of what they re able to do, in terms of what they re trying to do, in terms of how much they work, all that kind of stuff.
8
SPEAKER_00
Yeah.
8
SPEAKER_00
I mean, I will say Tesla gets a little too much bad rep for this, because what s happening is Tesla is a bursty environment.
8
SPEAKER_00
So I would say the baseline, my only point of reference is Google, where I ve interned three times, and I saw what it s like inside Google and DeepMind.
8
SPEAKER_00
I would say the baseline is higher than that.
8
SPEAKER_00
But then there s a punctuated equilibrium, where once in a while, there s a fire, and people work really hard.
8
SPEAKER_00
And so it s spiky and bursty.
8
SPEAKER_00
And then all the stories get collected.
8
SPEAKER_00
And then it gives the appearance of total insanity.
8
SPEAKER_00
But actually, it s just a bit more intense environment.
8
SPEAKER_00
And there are fires and sprints.
8
SPEAKER_00
And so I think, definitely though, I would say it s a more intense environment than something you would get at Google.
8
SPEAKER_01
In your personal, forget all of that, just in your own personal life, what do you think about the happiness of a human being, a brilliant person like yourself, about finding a balance between work and life, or is it such a thing, not a good thought experiment?
8
SPEAKER_00
Yeah, I think balance is good, but I also love to have sprints that are out of distribution.
8
SPEAKER_00
And that s when I think I ve been pretty creative as well.
8
SPEAKER_01
Sprints out of distribution means that most of the time you have a quote unquote balance.
8
SPEAKER_00
I have balance most of the time, but I like being obsessed with something once in a while.
8
SPEAKER_01
Once in a while is what, once a week, once a month, once a year?
8
SPEAKER_00
Yeah, probably like say once a month or something, yeah.
8
SPEAKER_01
And that s when we get a new GitHub repo for monitoring?
8
SPEAKER_00
Yeah, that s when you like really care about a problem.
8
SPEAKER_00
It must exist.
8
SPEAKER_00
This will be awesome.
8
SPEAKER_00
You re obsessed with it.
8
SPEAKER_00
And now you can t just do it on that day.
8
SPEAKER_00
You need to pay the fixed cost of getting into the groove.
8
SPEAKER_00
And then you need to stay there for a while.
8
SPEAKER_00
And then society will come and they will try to mess with you and they will try to distract you.
8
SPEAKER_00
Yeah, the worst thing is like a person who s like, I just need five minutes of your time.
8
SPEAKER_01
Yeah.
8
SPEAKER_00
This is, the cost of that is not five minutes.
8
SPEAKER_00
And society needs to change how it thinks about just five minutes of your time.
8
SPEAKER_01
Right.
8
SPEAKER_01
It s never, it s never just one minute.
8
SPEAKER_01
It s just 36, just a quick thing.
8
SPEAKER_01
What s the big deal?
8
SPEAKER_00
Why are you being so?
8
SPEAKER_01
Yeah, no.
8
SPEAKER_01
What s your computer setup?
8
SPEAKER_01
What s like the perfect, are you somebody that s flexible to no matter what, laptop, four screens?
8
SPEAKER_01
Or do you prefer a certain setup that you re most productive?
8
SPEAKER_00
I guess the one that I m familiar with is one large screen, 27 inch, and my laptop on the side.
8
SPEAKER_01
What operating system?
8
SPEAKER_00
I do Macs, that s my primary.
8
SPEAKER_01
For all tasks?
8
SPEAKER_00
I would say OSX, but when you re working on deep learning, everything is Linux.
8
SPEAKER_00
You re SSHed into a cluster and you re working remotely.
8
SPEAKER_01
But what about the actual development, like using the IDE?
8
SPEAKER_00
Yeah, you would use, I think a good way is you just run VS Code, my favorite editor right now, on your Mac, but you are actually, you have a remote folder through SSH.
8
SPEAKER_00
So the actual files that you re manipulating are on the cluster somewhere else.
8
SPEAKER_01
So what s the best IDE?
8
SPEAKER_01
VS Code, what else do people, so I use Emacs still.
8
SPEAKER_01
It may be cool, I don t know if it s maximum productivity.
8
SPEAKER_01
So what do you recommend in terms of editors?
8
SPEAKER_01
You worked with a lot of software engineers.
8
SPEAKER_01
Editors for Python, C++, machine learning applications?
8
SPEAKER_00
I think the current answer is VS Code.
8
SPEAKER_00
Currently, I believe that s the best IDE.
8
SPEAKER_00
It s got a huge amount of extensions.
8
SPEAKER_00
It has GitHub Copilot integration, which I think is very valuable.
8
SPEAKER_01
What do you think about the Copilot integration?
8
SPEAKER_01
I was actually, I got to talk a bunch with Guido Narasim, who s the creator of Python, and he loves Copilot.
8
SPEAKER_01
He programs a lot with it.
8
SPEAKER_01
Do you?
8
SPEAKER_00
Yeah, use Copilot.
8
SPEAKER_00
I love it.
8
SPEAKER_00
And it s free for me, but I would pay for it.
8
SPEAKER_00
Yeah, I think it s very good.
8
SPEAKER_00
And the utility that I found with it was, I would say there is a learning curve and you need to figure out when it s helpful and when to pay attention to its outputs and when it s not going to be helpful, where you should not pay attention to it.
8
SPEAKER_00
Because if you re just reading at suggestions all the time, it s not a good way of interacting with it.
8
SPEAKER_00
But I think I was able to mold myself to it.
8
SPEAKER_00
I find it s very helpful, number one, in copy-paste and replace some parts.
8
SPEAKER_00
So when the pattern is clear, it s really good at completing the pattern.
8
SPEAKER_00
And number two, sometimes it suggests APIs that I m not aware of.
8
SPEAKER_00
So it tells you about something that you didn t know.
8
SPEAKER_01
And that s an opportunity to discover a new API.
8
SPEAKER_00
It s an opportunity to... I would never take Copilot code as given.
8
SPEAKER_00
I almost always copy paste into a Google search and you see what this function is doing.
8
SPEAKER_00
And then you re like, oh, it s actually exactly what I need.
8
SPEAKER_00
Thank you, Copilot.
8
SPEAKER_00
So you learn something.
8
SPEAKER_01
So it s in part a search engine, a part maybe getting the exact syntax correctly that once you see it,
8
SPEAKER_01
It s that NP-hard thing.
8
SPEAKER_01
Once you see it, you know it s correct, but you yourself struggle.
8
SPEAKER_01
You can verify efficiently, but you can t generate efficiently.
8
SPEAKER_00
And Copilot really, I mean, it s autopilot for programming, right?
8
SPEAKER_00
And currently it s doing the link following, which is like the simple copy-paste and sometimes suggest, but over time it s going to become more and more autonomous.
8
SPEAKER_00
And so the same thing will play out in not just coding, but actually across many, many different things probably.
8
SPEAKER_01
But coding is an important one, right?
8
SPEAKER_01
Writing programs.
8
SPEAKER_01
How do you see the future of that developing, the program synthesis, like being able to write programs that are more and more complicated?
8
SPEAKER_01
Because right now, it s human supervised in interesting ways.
8
SPEAKER_01
It feels like the transition will be very painful.
8
SPEAKER_00
My mental model for it is the same thing will happen as with the autopilot.
8
SPEAKER_00
So currently he s doing lane following, he s doing some simple stuff.
8
SPEAKER_00
And eventually we ll be doing autonomy and people will have to intervene less and less.
8
SPEAKER_01
And those could be like testing mechanisms.
8
SPEAKER_01
Like if it writes a function and that function looks pretty damn correct, but how do you know it s correct?
8
SPEAKER_01
Because you re like getting lazier and lazier as a programmer.
8
SPEAKER_01
Like your ability to, because like little bugs, but I guess it won t make little mistakes.
8
SPEAKER_00
No, it will.
8
SPEAKER_00
Copilot will make off-by-one subtle bugs.
8
SPEAKER_00
It has done that to me.
8
SPEAKER_01
But do you think future systems will?
8
SPEAKER_01
Or is it really the off-by-one is actually a fundamental challenge of programming?
8
SPEAKER_00
In that case, it wasn t fundamental, and I think things can improve.
8
SPEAKER_00
But yeah, I think humans have to supervise.
8
SPEAKER_00
I am nervous about people not supervising what comes out and what happens to, for example, the proliferation of bugs in all of our systems.
8
SPEAKER_00
I m nervous about that, but I think there will probably be some other copilots for bug finding and stuff like that at some point, because there will be a lot more automation for... Oh, man.
8
SPEAKER_01
It s like a program, a copilot that generates a compiler
8
SPEAKER_01
one that does a linter, one that does a type checker.
8
SPEAKER_01
It s a committee of like a GPT sort of like.
8
SPEAKER_01
And then there ll be like a manager for the committee.
8
SPEAKER_01
And then there ll be somebody that says a new version of this is needed.
8
SPEAKER_01
We need to regenerate it.
8
SPEAKER_00
Yeah.
8
SPEAKER_00
There were 10 GPTs.
8
SPEAKER_00
They were forwarded and gave 50 suggestions.
8
SPEAKER_00
Another one looked at it and picked a few that they like.
8
SPEAKER_00
A bug one looked at it and it was like, it s probably a bug.
8
SPEAKER_00
They got re-ranked by some other thing.
8
SPEAKER_00
And then a final ensemble GPT comes in and is like, okay, given everything you guys have told me, this is probably the next token.
8
SPEAKER_01
you know, the feeling is the number of programmers in the world has been growing and growing very quickly.
8
SPEAKER_01
Do you think it s possible that it ll actually level out and drop to like a very low number with this kind of world?
8
SPEAKER_01
Because then you d be doing software 2.0 programming, and you ll be doing this kind of generation of copilot type systems programming, but you won t be doing the old school software 1.0 programming.
8
SPEAKER_00
I don t currently think that they re just going to replace human programmers.
8
SPEAKER_00
I m so hesitant saying stuff like this, right?
8
SPEAKER_00
Because this is going to be replayed in five years.
8
SPEAKER_01
No, it s going to show that this is where we thought, because I agree with you.
8
SPEAKER_01
I think we might be very surprised, right?
8
SPEAKER_01
Like, what are the next, what s your sense of where we re staying with language models?
8
SPEAKER_01
Like, does it feel like the beginning or the middle or the end?
8
SPEAKER_00
The beginning, 100%.
8
SPEAKER_00
I think the big question in my mind is for sure GPT will be able to program quite well, competently and so on.
8
SPEAKER_00
How do you steer the system?
8
SPEAKER_00
You still have to provide some guidance to what you actually are looking for.
8
SPEAKER_00
And so how do you steer it?
8
SPEAKER_00
And how do you say, how do you talk to it?
8
SPEAKER_00
How do you,
8
SPEAKER_00
Audit it and verify that what is done is correct.
8
SPEAKER_00
And how do you work with this?
8
SPEAKER_00
And it s as much not just an AI problem, but a UI UX problem.
8
SPEAKER_00
So beautiful, fertile ground for so much interesting work for VS Code++, where it s not just human programming anymore.
8
SPEAKER_00
It s amazing.
8
SPEAKER_01
Yeah, so you re interacting with the system.
8
SPEAKER_01
So not just one prompt, but it s iterative prompting.
8
SPEAKER_01
You re trying to figure out having a conversation with the system.
8
SPEAKER_01
That actually, I mean, to me, that s super exciting to have a conversation with the program I m running.
8
SPEAKER_00
Yeah, maybe at some point you re just conversing with it.
8
SPEAKER_00
It s like, okay, here s what I want to do.
8
SPEAKER_00
Actually, this variable, maybe it s not even that low level as a variable, but.
8
SPEAKER_01
You can also imagine like,
8
SPEAKER_01
can you translate this to C++ and back to Python?
8
SPEAKER_00
And back to- Yeah, it already kind of exists in some way.
8
SPEAKER_01
No, but just like doing it as part of the program experience, like, I think I d like to write this function in C++.
8
SPEAKER_01
Or like, you just keep changing for different programs, because they have different syntax.
8
SPEAKER_01
Maybe I want to convert this into a functional language.
8
SPEAKER_01
And so like, you get to become multilingual as a programmer and dance back and forth efficiently.
8
SPEAKER_00
I mean, I think the UX of it, though, is still very hard to think through, because it s not just about writing code on a page.
8
SPEAKER_00
You have an entire developer environment.
8
SPEAKER_00
You have a bunch of hardware on it.
8
SPEAKER_00
You have some environmental variables.
9
SPEAKER_01
You have some scripts that are running in a Chrome job.
9
SPEAKER_01
There s a lot going on to working with computers.
9
SPEAKER_01
And how do these systems set up environment flags and work across multiple machines and set up screen sessions and automate different processes?
9
SPEAKER_01
How all that works and is auditable by humans and so on is a massive question at the moment.
9
SPEAKER_00
You ve built Archive Sanity.
9
SPEAKER_00
What is Archive?
9
SPEAKER_00
And what is the future of academic research publishing that you would like to see?
9
SPEAKER_01
So Archive is this pre-print server.
9
SPEAKER_01
So if you have a paper, you can submit it for publication to journals or conferences, and then wait six months, and then maybe get a decision, pass or fail, or you can just upload it to Archive.
9
SPEAKER_01
And then people can tweet about it three minutes later, and then everyone sees it, everyone reads it, and everyone can profit from it in their own little ways.
9
SPEAKER_00
And you can cite it, and it has an official look to it.
9
SPEAKER_00
It feels like a publication process.
9
SPEAKER_00
It feels different than if you just put it in a blog post.
9
SPEAKER_01
Oh, yeah.
9
SPEAKER_01
Yeah.
9
SPEAKER_01
I mean, it s a paper and usually the bar is higher for something that you would expect on archive as opposed to something you would see in a blog post.
9
SPEAKER_00
Well, the culture created the bar because you could probably post a pretty crappy picture in the archive.
9
SPEAKER_00
So what s that make you feel like?
9
SPEAKER_00
What s that make you feel about peer review?
9
SPEAKER_00
So rigorous peer review by two, three experts versus the peer review of the community right as it s written.
9
SPEAKER_01
Yeah, basically I think the community is very well able to peer review things very quickly on Twitter.
9
SPEAKER_01
And I think maybe it just has to do something with AI machine learning field specifically though.
9
SPEAKER_01
I feel like things are more easily auditable and the verification is easier potentially than the verification somewhere else.
9
SPEAKER_01
So it s kind of like, you can think of these scientific publications as like little blockchains where everyone s building on each other s work and studying each other.
9
SPEAKER_01
And you sort of have AI, which is kind of like this much faster and loose blockchain.
9
SPEAKER_01
But then you have, and any one individual entry is like very, very cheap to make.
9
SPEAKER_01
And then you have other fields where maybe that model doesn t make as much sense.
9
SPEAKER_01
And so I think in AI, at least, things are pretty easily verifiable.
9
SPEAKER_01
And so that s why when people upload papers that are a really good idea and so on, people can try it out the next day.
9
SPEAKER_01
And they can be the final arbiter of whether it works or not on their problem.
9
SPEAKER_01
And the whole thing just moves significantly faster.
9
SPEAKER_01
So I kind of feel like academia still has a place.
9
SPEAKER_01
So this conference journal process still has a place.
9
SPEAKER_01
But it s sort of like it lags behind, I think.
9
SPEAKER_01
And it s a bit more maybe higher quality process, but it s not sort of the place where you will discover cutting edge work anymore.
9
SPEAKER_01
It used to be the case when I was starting my PhD that you go to conferences and journals and you discuss all the latest research.
9
SPEAKER_01
Now, when you go to a conference or journal, like no one discusses anything that s there because it s already like three generations ago.
9
SPEAKER_01
Irrelevant.
9
SPEAKER_00
Yeah, which makes me sad about like DeepMind, for example, where they still publish in Nature and these big prestigious, I mean, there s still value, I suppose, to the prestige that comes with these big venues, but the result is that they ll announce some breakthrough performance, and it ll take like a year to actually publish the details.
9
SPEAKER_00
And those details, if they were published immediately, would inspire the community to move in certain directions.
9
SPEAKER_01
Yeah, it would speed up the rest of the community, but I don t know to what extent that s part of their objective function also.
9
SPEAKER_00
That s true.
9
SPEAKER_00
So it s not just the prestige, a little bit of the delay is part of it.
9
SPEAKER_01
Yeah, they certainly, DeepMind specifically, has been working in the regime of having slightly higher quality, basically process and latency, and publishing those papers that way.
9
SPEAKER_00
Another question from Reddit.
9
SPEAKER_00
Do you or have you suffered from imposter syndrome?
9
SPEAKER_00
Being the director of AI at Tesla, being this person when you re at Stanford where like the world looks at you as the expert in AI to teach the world about machine learning.
9
SPEAKER_01
I was leaving Tesla after five years, I spent a ton of time in meeting rooms, and I would read papers.
9
SPEAKER_01
In the beginning, when I joined Tesla, I was writing code, and then I was writing less and less code, and I was reading code, and then I was reading less and less code.
9
SPEAKER_01
This is just a natural progression that happens, I think.
9
SPEAKER_01
Definitely, I would say near the tail end, that s when it starts to hit you a bit more that you re supposed to be an expert.
9
SPEAKER_01
But actually,
9
SPEAKER_01
the source of truth is the code that people are writing, the GitHub, and the actual code itself.
9
SPEAKER_01
And you re not as familiar with that as you used to be.
9
SPEAKER_01
And so I would say maybe there s some insecurity there.
9
SPEAKER_00
Yeah, that s actually pretty profound, that a lot of the insecurity has to do with not writing the code in the computer science space.
9
SPEAKER_00
Because that is the truth, that right there.
9
SPEAKER_01
The code is the source of truth.
9
SPEAKER_01
The papers and everything else, it s a high-level summary.
9
SPEAKER_01
Yeah, it s just a high-level summary, but at the end of the day, you have to read code.
9
SPEAKER_01
It s impossible to translate all that code into actual paper form.
9
SPEAKER_01
So when things come out, especially when they have a source code available, that s my favorite place to go.
9
SPEAKER_00
So like I said, you re one of the greatest teachers of machine learning, AI, ever, from CS231N to today.
9
SPEAKER_00
What advice would you give to beginners interested in getting into machine learning?
9
SPEAKER_01
Beginners are often focused on like what to do.
9
SPEAKER_01
And I think the focus should be more like how much you do.
9
SPEAKER_01
So I am kind of like believer on a high level in this 10,000 hours kind of concept where you just kind of have to just pick the things where you can spend time and you care about and you re interested in.
9
SPEAKER_01
You literally have to put in 10,000 hours of work.
9
SPEAKER_01
It doesn t even matter as much where you put it, and you ll iterate, and you ll improve, and you ll waste some time.
9
SPEAKER_01
I don t know if there s a better way.
9
SPEAKER_01
You need to put in 10,000 hours.
9
SPEAKER_01
But I think it s actually really nice because I feel like there s some sense of determinism about being an expert at a thing if you spend 10,000 hours.
9
SPEAKER_01
You can literally pick an arbitrary thing, and I think if you spend 10,000 hours of deliberate effort and work, you actually will become an expert at it.
9
SPEAKER_01
And so I think that s kind of like a nice thought.
9
SPEAKER_01
And so basically I would focus more on like, are you spending 10,000 hours?
9
SPEAKER_01
That s what I d focus on.
9
SPEAKER_00
So, and then thinking about what kind of mechanisms maximize your likelihood of getting to 10,000 hours.
9
SPEAKER_00
Exactly.
9
SPEAKER_00
Which for us silly humans means probably forming a daily habit.
9
SPEAKER_00
of like every single day actually doing the thing.
9
SPEAKER_01
Whatever helps you.
9
SPEAKER_01
So I do think to a large extent it s a psychological problem for yourself.
9
SPEAKER_01
One other thing that I think is helpful for the psychology of it is many times people compare themselves to others in the area.
9
SPEAKER_01
I think this is very harmful.
9
SPEAKER_01
Only compare yourself to you from some time ago, like say a year ago.
9
SPEAKER_01
Are you better than you a year ago?
9
SPEAKER_01
It s the only way to think.
9
SPEAKER_01
And I think this, then you can see your progress and it s very motivating.
9
SPEAKER_00
That s so interesting that focus on the quantity of hours.
9
SPEAKER_00
I think a lot of people in the beginner stage, but actually throughout, get paralyzed.
9
SPEAKER_00
by the choice, like which one, do I pick this path or this path?
9
SPEAKER_00
Like they ll literally get paralyzed by like which IDE to use.
9
SPEAKER_01
Well, they re worried, yeah, they re worried about all these things, but the thing is, you will waste time doing something wrong.
9
SPEAKER_01
You will eventually figure out it s not right.
9
SPEAKER_01
You will accumulate scar tissue, and next time you ll grow stronger, because next time you ll have the scar tissue, and next time you ll learn from it, and now next time you come to a similar situation, you ll be like, oh,
9
SPEAKER_01
messed up.
9
SPEAKER_01
I ve spent a lot of time working on things that never materialize into anything.
9
SPEAKER_01
And I have all that scar tissue and I have some intuitions about what was useful, what wasn t useful, how things turned out.
9
SPEAKER_01
So all those mistakes were not dead work.
9
SPEAKER_01
So I just think you should just focus on working.
9
SPEAKER_01
What have you done?
9
SPEAKER_01
What have you done last week?
9
SPEAKER_00
That s a good question actually to ask for a lot of things, not just machine learning.
9
SPEAKER_00
It s a good way to cut the, I forgot what the term we used, but the fluff, the blubber, whatever the inefficiencies in life.
9
SPEAKER_00
What do you love about teaching?
9
SPEAKER_00
You seem to find yourself often in the, like drawn to teaching.
9
SPEAKER_00
You re very good at it, but you re also drawn to it.
9
SPEAKER_01
I mean, I don t think I love teaching.
9
SPEAKER_01
I love happy humans.
9
SPEAKER_01
And happy humans like when I teach.
9
SPEAKER_01
I wouldn t say I hate teaching, I tolerate teaching, but it s not like the act of teaching that I like.
9
SPEAKER_01
It s that I have something, I m actually okay at it.
9
SPEAKER_01
I m okay at teaching and people appreciate it a lot.
9
SPEAKER_01
And so I m just happy to try to be helpful.
9
SPEAKER_01
And teaching itself is not like the most, I mean, it can be really annoying, frustrating.
9
SPEAKER_01
I was working on a bunch of lectures just now.
9
SPEAKER_01
I was reminded back to my days of 231N just how much work it is to create some of these materials and make them good.
9
SPEAKER_01
The amount of iteration and thought and you go down blind alleys and just how much you change it.
9
SPEAKER_01
So creating something good in terms of like educational value is really hard.
9
SPEAKER_01
And it s not fun.
9
SPEAKER_00
It was difficult, so people should definitely go watch your new stuff you put out.
9
SPEAKER_00
There are lectures where you re actually building the thing from, like you said, the code is truth.
9
SPEAKER_00
So discussing back propagation by building it, by looking through it, just the whole thing.
9
SPEAKER_00
So how difficult is that to prepare for?
9
SPEAKER_00
I think that s a really powerful way to teach.
9
SPEAKER_00
Did you have to prepare for that, or are you just live thinking through it?
9
SPEAKER_01
I will typically do like say three takes and then I take like the better take.
9
SPEAKER_01
So I do multiple takes and I take some of the better takes and then I just build out a lecture that way.
9
SPEAKER_01
Sometimes I have to delete 30 minutes of content because it just went down an alley that I didn t like too much.
9
SPEAKER_01
So there s a bunch of iteration and it probably takes me, you know, somewhere around 10 hours to create one hour of content.
9
SPEAKER_01
To get one hour.
9
SPEAKER_00
It s interesting, I mean, is it difficult to go back to the basics?
9
SPEAKER_00
Do you draw a lot of wisdom from going back to the basics?
9
SPEAKER_01
Yeah, going back to back propagation loss functions, where they come from.
9
SPEAKER_01
And one thing I like about teaching a lot, honestly, is it definitely strengthens your understanding.
9
SPEAKER_01
So, it s not a purely altruistic activity.
9
SPEAKER_01
It s a way to learn.
9
SPEAKER_01
If you have to explain something to someone, you realize you have gaps in knowledge.
9
SPEAKER_01
And so, I even surprised myself in those lectures.
9
SPEAKER_01
Like, the result will obviously look like this, and then the result doesn t look like it.
9
SPEAKER_01
And I m like, okay, I thought I understood this.
9
SPEAKER_00
But that s why it s really cool to literally code.
9
SPEAKER_00
You run it in a notebook and it gives you a result and you re like, oh, wow.
9
SPEAKER_00
And like actual numbers, actual input, actual code.
9
SPEAKER_01
Yeah, it s not mathematical symbols, et cetera.
9
SPEAKER_01
The source of truth is the code.
9
SPEAKER_01
It s not slides.
9
SPEAKER_01
It s just like, let s build it.
9
SPEAKER_00
It s beautiful.
9
SPEAKER_00
You re a rare human in that sense.
9
SPEAKER_00
What advice would you give to researchers trying to develop and publish idea that have a big impact in the world of AI?
9
SPEAKER_00
So maybe undergrads, maybe early graduate students.
9
SPEAKER_01
Yeah.
9
SPEAKER_01
I mean, I would say they definitely have to be a little bit more strategic than I had to be as a PhD student because of the way AI is evolving.
9
SPEAKER_01
It s going the way of physics, where in physics, you used to be able to do experiments on your benchtop, and everything was great, and you could make progress.
9
SPEAKER_01
And now you have to work in LHC or CERN.
9
SPEAKER_01
And so AI is going in that direction as well.
9
SPEAKER_01
So there s certain kinds of things that s just not possible to do on the benchtop anymore.
9
SPEAKER_01
I think that didn t used to be the case at the time.
9
SPEAKER_00
Do you still think that there s like GAN-type papers to be written, or like very simple idea that requires just one computer to illustrate a simple example?
9
SPEAKER_01
I mean, one example that s been very influential recently is diffusion models.
9
SPEAKER_01
diffusion models are amazing.
9
SPEAKER_01
Diffusion models are six years old.
9
SPEAKER_01
For the longest time, people were kind of ignoring them, as far as I can tell.
9
SPEAKER_01
And they re an amazing generative model, especially in images.
9
SPEAKER_01
And so stable diffusion and so on, it s all diffusion-based.
9
SPEAKER_01
Diffusion is new.
9
SPEAKER_01
It was not there and came from Google, but a researcher could have come up with it.
9
SPEAKER_01
In fact, some of the first
9
SPEAKER_01
Actually, no, those came from Google as well.
9
SPEAKER_01
But a researcher could come up with that in an academic institution.
9
SPEAKER_00
Yeah, what do you find most fascinating about diffusion models?
9
SPEAKER_00
So, from the societal impact to the technical architecture.
9
SPEAKER_01
What I like about diffusion is it works so well.
9
SPEAKER_00
Is that surprising to you?
9
SPEAKER_00
The amount of, the variety, almost the novelty of the synthetic data it s generating?
9
SPEAKER_01
Yeah, so the stable diffusion images are incredible.
9
SPEAKER_01
The speed of improvement in generating images has been insane.
9
SPEAKER_01
We went very quickly from generating tiny digits to tiny faces, and it all looked messed up, and now we have stable diffusion.
9
SPEAKER_01
And that happened very quickly.
9
SPEAKER_01
There s a lot that academia can still contribute.
9
SPEAKER_01
You know, for example,
9
SPEAKER_01
Flash attention is a very efficient kernel for running the attention operation inside a transformer that came from academic environment.
9
SPEAKER_01
It s a very clever way to structure the kernel that does the calculation, so it doesn t materialize the attention matrix.
9
SPEAKER_01
And so there s, I think there s still like lots of things to contribute, but you have to be just more strategic.
9
SPEAKER_00
Do you think neural networks can be made to reason?
9
SPEAKER_01
Yes.
9
SPEAKER_00
Do you think they already reason?
9
SPEAKER_01
Yes.
9
SPEAKER_00
What s your definition of reasoning?
9
SPEAKER_01
information processing.
9
SPEAKER_00
So in the way that humans think through a problem and come up with novel ideas, it feels like reasoning.
9
SPEAKER_00
So the novelty, I don t want to say, but out of distribution ideas, you think it s possible?
9
SPEAKER_01
Yes.
9
SPEAKER_01
And I think we re seeing that already in the current neural nets.
9
SPEAKER_01
You re able to remix the training set information into true generalization in some sense.
9
SPEAKER_00
That doesn t appear.
9
SPEAKER_01
It doesn t appear verbatim in the training set.
9
SPEAKER_01
Like you re doing something interesting algorithmically.
9
SPEAKER_01
You re manipulating, you know, some symbols and you re coming up with some correct, unique answer in a new setting.
9
SPEAKER_00
What would illustrate to you, holy shit, this thing is definitely thinking?
9
SPEAKER_01
To me, thinking or reasoning is just information processing and generalization.
9
SPEAKER_01
And I think the neural nets already do that today.
9
SPEAKER_00
So being able to perceive the world or perceive whatever the inputs are and to make predictions based on that or actions based on that, that s reasoning.
9
SPEAKER_01
Yeah, you re giving correct answers in novel settings by manipulating information.
9
SPEAKER_01
You ve learned the correct algorithm.
9
SPEAKER_01
You re not doing just some kind of a lookup table and there s neighbor search.
9
SPEAKER_00
something like that.
9
SPEAKER_00
Let me ask you about AGI.
9
SPEAKER_00
What are some moonshot ideas you think might make significant progress towards AGI?
9
SPEAKER_00
And maybe in other ways, what are the big blockers that we re missing now?
9
SPEAKER_01
So basically, I am fairly bullish on our ability to build AGIs, basically automated systems that we can interact with and are very human-like.
9
SPEAKER_01
And we can interact with them in a digital realm or a physical realm.
9
SPEAKER_01
Currently, it seems most of the models that sort of do these sort of magical tasks are in a text realm.
9
SPEAKER_01
I think, as I mentioned, I m suspicious that the text realm is not enough to actually build full understanding of the world.
9
SPEAKER_01
I do actually think you need to go into pixels and understand the physical world and how it works.
9
SPEAKER_01
So I do think that we need to extend these models to consume images and videos and train on a lot more data that is multimodal in that way.
9
SPEAKER_01
Do you think you need to touch the world to understand it also?
9
SPEAKER_01
Well, that s the big open question I would say in my mind, is if you also require the embodiment and the ability to sort of interact with the world, run experiments, and have the data of that form, then you need to go to Optimus.
9
SPEAKER_01
or something like that.
9
SPEAKER_01
And so I would say Optimus in some way is like a hedge in AGI because it seems to me that it s possible that just having data from the internet is not enough.
9
SPEAKER_01
If that is the case, then Optimus may lead to AGI.
9
SPEAKER_01
Because Optimus, to me, there s nothing beyond Optimus.
9
SPEAKER_01
You have this humanoid form factor that can actually do stuff in the world.
9
SPEAKER_01
You can have millions of them interacting with humans and so on.
9
SPEAKER_01
And if that doesn t give rise to AGI at some point, I m not sure what will.
9
SPEAKER_01
So from a completeness perspective, I think that s a really good platform, but it s a much harder platform because you are dealing with atoms and you need to actually like build these things and integrate them into society.
9
SPEAKER_01
So I think that path takes longer, but it s much more certain.
9
SPEAKER_01
And then there s a path of the internet and just like training these compression models effectively on trying to compress all the internet.
9
SPEAKER_01
And that might also give these agents as well.
9
SPEAKER_00
compress the internet, but also interact with the internet.
9
SPEAKER_00
So it s not obvious to me.
9
SPEAKER_00
In fact, I suspect you can reach AGI without ever entering the physical world, which is a little bit more concerning because that results in it happening faster.
9
SPEAKER_00
So it just feels like we re in boiling water.
9
SPEAKER_00
We won t know as it s happening.
9
SPEAKER_00
I would like to, I m not afraid of AGI, I m excited about it, there s always concerns, but I would like to know when it happens.
9
SPEAKER_01
Yeah.
9
SPEAKER_00
And have like hints about when it happens, like a year from now it will happen, that kind of thing.
9
SPEAKER_01
Yeah.
9
SPEAKER_00
I just feel like in the digital realm, it just might happen.
9
SPEAKER_01
Yeah.
9
SPEAKER_01
I think all we have available to us, because no one has built AGI again, so all we have available to us is, is there enough fertile ground on the periphery?
9
SPEAKER_01
I would say yes.
9
SPEAKER_01
And we have the progress so far, which has been very rapid.
9
SPEAKER_01
And there are next steps that are available.
9
SPEAKER_01
And so I would say, yeah, it s quite likely that we ll be interacting with digital entities
9
SPEAKER_00
How will you know that somebody...
10
SPEAKER_00
is built AGI?
10
SPEAKER_00
It s going to be a slow, I think it s going to be a slow incremental transition.
10
SPEAKER_00
It s going to be product based and focused.
10
SPEAKER_00
It s going to be GitHub Copilot getting better.
10
SPEAKER_00
And then GPT is helping you write.
10
SPEAKER_00
And then these oracles that you can go to with mathematical problems.
10
SPEAKER_00
I think we re on a verge of being able to ask very complex questions in chemistry, physics, math of these oracles and have them complete solutions.
10
SPEAKER_01
So AGI to use primarily focus on intelligence.
10
SPEAKER_01
So consciousness doesn t enter
10
SPEAKER_01
into it?
10
SPEAKER_00
So, in my mind, consciousness is not a special thing you will figure out and bolt on.
10
SPEAKER_00
I think it s an emergent phenomenon of a large enough and complex enough generative model, sort of.
10
SPEAKER_00
So, if you have a complex enough world model that understands the world, then it also understands its predicament in the world as being a language model, which to me is a form of consciousness or self-awareness and
10
SPEAKER_01
So in order to understand the world deeply, you probably have to integrate yourself into the world.
10
SPEAKER_01
And in order to interact with humans and other living beings, consciousness is a very useful tool.
10
SPEAKER_00
I think consciousness is like a modeling insight.
10
SPEAKER_01
Modeling insight.
10
SPEAKER_00
Yeah, it s a, you have a powerful enough model of understanding the world that you actually understand that you are an entity in it.
10
SPEAKER_01
Yeah, but there s also this, perhaps just the narrative we tell ourselves, there s a, it feels like something to experience the world.
10
SPEAKER_01
The hard problem of consciousness.
10
SPEAKER_00
Yeah.
10
SPEAKER_01
But that could be just a narrative that we tell ourselves.
10
SPEAKER_00
Yeah, I think it will emerge.
10
SPEAKER_00
I think it s going to be something very boring.
10
SPEAKER_00
Like, we ll be talking to these digital AIs.
10
SPEAKER_00
They will claim they re conscious.
10
SPEAKER_00
They will appear conscious.
10
SPEAKER_00
They will do all the things that you would expect of other humans.
10
SPEAKER_00
And it s going to just be a stalemate.
10
SPEAKER_01
I think there ll be a lot of actual fascinating ethical questions, like Supreme Court level questions.
10
SPEAKER_01
of whether you re allowed to turn off a conscious AI, if you re allowed to build a conscious AI, maybe there would have to be the same kind of debates that you have around.
10
SPEAKER_01
Sorry to bring up a political topic, but abortion, which is the deeper question with abortion, is what is life?
10
SPEAKER_01
And the deep question with AI is also, what is life?
10
SPEAKER_01
And what is conscious?
10
SPEAKER_01
And I think that ll be very fascinating.
10
SPEAKER_01
to bring up, it might become illegal to build systems that are capable of such level of intelligence that consciousness would emerge and therefore the capacity to suffer would emerge.
10
SPEAKER_01
A system that says, no, please don t kill me.
10
SPEAKER_00
Well, that s what the Lambda chatbot already told this Google engineer, right?
10
SPEAKER_00
It was talking about not wanting to die or so on.
10
SPEAKER_01
So that might become illegal to do that.
10
SPEAKER_00
Right.
10
SPEAKER_01
Because otherwise you might have a lot of creatures that don t wanna die, and they will... You can just spawn infinity of them on a cluster.
10
SPEAKER_01
And then that might lead to horrible consequences, because then there might be a lot of people that secretly love murder, and they ll start practicing murder on those systems.
10
SPEAKER_01
To me, all of this stuff just brings a beautiful mirror to the human condition, and human nature will get to explore it.
10
SPEAKER_01
And that s what the best,
10
SPEAKER_01
of the Supreme Court, of all the different debates we have about ideas of what it means to be human.
10
SPEAKER_01
We get to ask those deep questions that we ve been asking throughout human history.
10
SPEAKER_01
There s always been the other in human history.
10
SPEAKER_01
We re the good guys and that s the bad guys and we re going to, you know, throughout human history, let s murder the bad guys.
10
SPEAKER_01
And the same will probably happen with robots.
10
SPEAKER_01
It ll be the other at first, and then we ll get to ask questions.
10
SPEAKER_01
What does it mean to be alive?
10
SPEAKER_01
What does it mean to be conscious?
10
SPEAKER_00
Yeah.
10
SPEAKER_00
And I think there s some canary in the coal mines, even with what we have today.
10
SPEAKER_00
And, you know, for example, there s these waifus that you can work with, and some people are trying to, like, this company is going to shut down, but this person really loved their waifu, and is trying to port it somewhere else, and it s not possible.
10
SPEAKER_00
I think like definitely people will have feelings towards these systems because in some sense they are like a mirror of humanity because they are like sort of like a big average of humanity in the way that it s trained.
10
SPEAKER_01
But we can, that average, we can actually watch.
10
SPEAKER_01
It s nice to be able to interact with the big average of humanity and do like a search query on it.
10
SPEAKER_00
Yeah, yeah, it s very fascinating.
10
SPEAKER_00
And we can also, of course, also like shape it.
10
SPEAKER_00
It s not just a pure average.
10
SPEAKER_00
We can mess with the training data.
10
SPEAKER_00
We can mess with the objective.
10
SPEAKER_00
We can fine tune them in various ways.
10
SPEAKER_00
So we have some, you know, impact on what those systems look like.
10
SPEAKER_01
If you want to achieve AGI and you could have a conversation with her and ask her, talk about anything, maybe ask her a question, what kind of stuff would you ask?
10
SPEAKER_00
I would have some practical questions in my mind like, do I or my loved ones really have to die?
10
SPEAKER_00
What can we do about that?
10
SPEAKER_00
Do you think it will answer clearly or would it answer poetically?
10
SPEAKER_00
I would expect it to give solutions.
10
SPEAKER_00
I would expect it to be like, well, I ve read all of these textbooks, and I know all these things that you ve produced, and it seems to me like here are the experiments that I think it would be useful to run next, and here are some gene therapies that I think would be helpful, and here are the kinds of experiments that you should run.
10
SPEAKER_01
Okay, let s go with this thought experiment, okay?
10
SPEAKER_01
Imagine that
10
SPEAKER_01
Mortality is actually like a prerequisite for happiness.
10
SPEAKER_01
So if we become immortal, we ll actually become deeply unhappy.
10
SPEAKER_01
And the model is able to know that.
10
SPEAKER_01
So what is it supposed to tell you, stupid human, about it?
10
SPEAKER_01
Yes, you can become immortal, but you ll become deeply unhappy.
10
SPEAKER_01
If the AGI system is trying to empathize with you human, what is it supposed to tell you?
10
SPEAKER_01
That yes, you don t have to die, but you re really not gonna like it?
10
SPEAKER_01
Is it gonna be deeply honest?
10
SPEAKER_01
There s a interstellar, what is it?
10
SPEAKER_01
The AI says humans want 90% honesty.
10
SPEAKER_01
So you have to pick how honest do I wanna answer these practical questions.
10
SPEAKER_00
I love AI Interstellar, by the way.
10
SPEAKER_00
I think it s such a sidekick to the entire story, but at the same time, it s really interesting.
10
SPEAKER_00
It s kind of limited in certain ways, right?
10
SPEAKER_00
Yeah, it s limited.
10
SPEAKER_00
And I think that s totally fine, by the way.
10
SPEAKER_00
I think it s fine and plausible to have limited and imperfect AGIs.
10
SPEAKER_01
Is that a feature almost?
10
SPEAKER_00
As an example, it has a fixed amount of compute on its physical body, and it might just be that even though you can have a super amazing mega brain, super intelligent AI, you also can have less intelligent AIs that you can deploy in a power efficient way, and then they re not perfect, they might make mistakes.
10
SPEAKER_01
No, I meant more like, say you had infinite compute, and it s still good to make mistakes sometimes.
10
SPEAKER_01
In order to integrate yourself,
10
SPEAKER_01
What is it, going back to Good Will Hunting, Robin Williams  character says like the human imperfections, that s the good stuff, right?
10
SPEAKER_01
Isn t that the, like we don t want perfect, we want flaws in part to form connections with each other, because it feels like something you can attach your feelings to, the flaws.
10
SPEAKER_01
And in that same way, you want an AI that s flawed.
10
SPEAKER_01
I don t know.
10
SPEAKER_01
I feel like perfection is cool.
10
SPEAKER_01
But that s not AGI.
10
SPEAKER_01
But see, AGI would need to be intelligent enough to give answers to humans that humans don t understand.
10
SPEAKER_01
And I think perfect is something humans can t understand.
10
SPEAKER_01
Because even science doesn t give perfect answers.
10
SPEAKER_01
There s always gaps and mysteries and I don t know.
10
SPEAKER_01
I don t know if humans want perfect.
10
SPEAKER_00
Yeah, I could imagine just having a conversation with this kind of oracle entity, as you d imagine them, and yeah, maybe it can tell you about, based on my analysis of human condition, you might not want this, and here are some of the things that might matter.
10
SPEAKER_01
But every dumb human will say, yeah, yeah, yeah, yeah, yeah, trust me, give me the truth, I can handle it.
10
SPEAKER_00
But that s the beauty, people can choose.
10
SPEAKER_01
But then...
10
SPEAKER_01
The old marshmallow test with the kids and so on.
10
SPEAKER_01
I feel like too many people.
10
SPEAKER_01
like can t handle the truth, probably including myself, like the deep truth of the human condition, I don t know if I can handle it.
10
SPEAKER_01
Like what if there s some darks, what if we are an alien science experiment and it realizes that, what if it had, I mean.
10
SPEAKER_01
I mean, this is the matrix all over again.
10
SPEAKER_01
I don t know, I would, what would I talk about?
10
SPEAKER_01
I don t even, yeah, I,
10
SPEAKER_01
Probably I ll go with the safer scientific questions at first that have nothing to do with my own personal life and mortality, just like about physics and so on.
10
SPEAKER_01
To build up, let s see where it s at, or maybe see if it has a sense of humor.
10
SPEAKER_01
That s another question.
10
SPEAKER_01
Would it be able to, presumably in order to, if it understands humans deeply, would it be able to generate
10
SPEAKER_01
to generate humor.
10
SPEAKER_00
Yeah, I think that s actually a wonderful benchmark almost.
10
SPEAKER_00
I think that s a really good point basically.
10
SPEAKER_00
To make you laugh.
10
SPEAKER_00
Yeah, if it s able to be like a very effective stand-up comedian that is doing something very interesting computationally.
10
SPEAKER_00
I think being funny is extremely hard.
10
SPEAKER_01
Yeah.
10
SPEAKER_01
Because it s hard in a way, like a Turing test, the original intent of the Turing test is hard, because you have to convince humans.
10
SPEAKER_01
And there s nothing, that s why comedians talk about this, like this is deeply honest.
10
SPEAKER_01
Because if people can t help but laugh, and if they don t laugh, that means you re not funny.
10
SPEAKER_01
If they laugh, it s funny.
10
SPEAKER_00
And you re showing, you need a lot of knowledge to create humor, about like, like you mentioned, human condition and so on, and then you need to be clever with it.
10
SPEAKER_01
You mentioned a few movies, you tweeted, movies that I ve seen five plus times but am ready and willing to keep watching, Interstellar, Gladiator, Contact, Good Will Hunting, The Matrix, Lord of the Rings, all three, Avatar, Fifth Element, so on, it goes on, Terminator 2, Mean Girls, I m not gonna ask about that one.
10
SPEAKER_00
I think her man.
10
SPEAKER_00
Mean Girls is great.
10
SPEAKER_01
What are some that jump out to you in your memory that you love
10
SPEAKER_01
And why?
10
SPEAKER_01
You mentioned the Matrix.
10
SPEAKER_01
As a computer person, why do you love the Matrix?
10
SPEAKER_00
There s so many properties that make it beautiful and interesting.
10
SPEAKER_00
So there s all these philosophical questions, but then there s also AGIs, and there s simulation, and it s cool, and there s the black.
10
SPEAKER_00
The look of it, the feel of it.
10
SPEAKER_00
Yeah, the look of it, the feel of it, the action, the bullet time.
10
SPEAKER_00
It was just innovating in so many ways.
10
SPEAKER_01
And then Good Will Hunting, why do you like that one?
10
SPEAKER_00
Yeah, I really like this tortured genius sort of character who s grappling with whether or not he has any responsibility or what to do with this gift that he was given or how to think about the whole thing.
10
SPEAKER_01
There s also a dance between the genius and the personal, what it means to love another human being.
10
SPEAKER_00
There s a lot of themes there.
10
SPEAKER_00
It s just a beautiful movie.
10
SPEAKER_01
And then the fatherly figure, the mentor and the psychiatrist.
10
SPEAKER_00
It really messes with you.
10
SPEAKER_00
There s some movies that just really mess with you on a deep level.
10
SPEAKER_01
Do you relate to that movie at all?
10
SPEAKER_00
No.
10
SPEAKER_01
It s not your fault, Andre, as I said.
10
SPEAKER_01
Lord of the Rings, that s self-explanatory.
10
SPEAKER_01
Terminator 2, which is interesting.
10
SPEAKER_01
You rewatch that a lot.
10
SPEAKER_01
Is that better than Terminator 1?
10
SPEAKER_01
You like Arnold?
10
SPEAKER_00
I do like Terminator 1 as well.
10
SPEAKER_00
I like Terminator 2 a little bit more, but in terms of like its surface properties.
10
SPEAKER_01
Do you think Skynet is at all a possibility?
10
SPEAKER_01
Yes.
10
SPEAKER_01
Like the actual sort of autonomous weapon system kind of thing?
10
SPEAKER_01
Do you worry about that stuff?
10
SPEAKER_00
I do worry about it 100%.
10
SPEAKER_00
I 100% worry about it.
10
SPEAKER_00
Some of these fears of AGIs and how this will play out, these will be very powerful entities probably at some point.
10
SPEAKER_00
For a long time, they re going to be tools in the hands of humans.
10
SPEAKER_00
People talk about alignment of AGIs and how to make – the problem is even humans are not aligned.
10
SPEAKER_00
how this will be used and what this is gonna look like is, yeah, it s troubling, so.
10
SPEAKER_01
Do you think it ll happen slowly enough that we ll be able to, as a human civilization, think through the problems?
10
SPEAKER_00
Yes, that s my hope, is that it happens slowly enough and in an open enough way where a lot of people can see and participate in it.
10
SPEAKER_00
just figure out how to deal with this transition, I think, where it was gonna be interesting.
10
SPEAKER_01
I draw a lot of inspiration from nuclear weapons, because I sure thought it would be fucked once they developed nuclear weapons.
10
SPEAKER_01
But it s almost like when the systems are not so dangerous, they destroy human civilization, we deploy them and learn the lessons, and then we quickly
10
SPEAKER_01
If it s too dangerous, we ll quickly, quickly, we might still deploy it, but you very quickly learn not to use them.
10
SPEAKER_01
And so there ll be like this balance achieved.
10
SPEAKER_01
Humans are very clever as a species.
10
SPEAKER_01
It s interesting.
10
SPEAKER_01
We exploit the resources as much as we can, but we don t, we avoid destroying ourselves, it seems like.
10
SPEAKER_00
Well, I don t know about that, actually.
10
SPEAKER_01
I hope it continues.
10
SPEAKER_00
I mean, I m definitely concerned about nuclear weapons and so on, not just as a result of the recent conflict, even before that.
10
SPEAKER_00
That s probably my number one concern for humanity.
10
SPEAKER_01
So if humanity destroys itself or destroys 90% of people, that would be because of nukes?
10
SPEAKER_00
I think so.
10
SPEAKER_00
And it s not even about the full destruction.
10
SPEAKER_00
To me, it s bad enough if we reset society.
10
SPEAKER_00
That would be terrible.
10
SPEAKER_00
That would be really bad.
10
SPEAKER_00
And I can t believe we re so close to it.
10
SPEAKER_00
It s so crazy to me.
10
SPEAKER_01
It feels like we might be a few tweets away from something like that.
10
SPEAKER_00
Yup.
10
SPEAKER_00
Basically, it s extremely unnerving and has been for me for a long time.
10
SPEAKER_01
It seems unstable that world leaders just having a bad mood
10
SPEAKER_01
can take one step towards a bad direction and it escalates.
10
SPEAKER_01
And because of a collection of bad moods, it can escalate without being able to stop.
10
SPEAKER_00
It s a huge amount of power and then also with the proliferation.
10
SPEAKER_00
Basically, I don t actually know what the good outcomes are here.
10
SPEAKER_00
So I m definitely worried about it a lot.
10
SPEAKER_00
And then AGI is not currently there, but I think at some point will more and more become something like it.
10
SPEAKER_00
The danger with AGI even is that I think it s even like slightly worse in a sense that
10
SPEAKER_00
there are good outcomes of AGI, and then the bad outcomes are like an epsilon away, like a tiny one away.
10
SPEAKER_00
And so I think capitalism and humanity and so on will drive for the positive ways of using that technology, but then if bad outcomes are just like a tiny, like flip a minus sign away, that s a really bad position to be in.
10
SPEAKER_01
A tiny perturbation of the system results in the destruction of the human species.
10
SPEAKER_01
It s a weird line to walk.
10
SPEAKER_00
Yeah, I think in general, what s really weird about the dynamics of humanity in this explosion we talked about is just the insane coupling afforded by technology and just the instability of the whole dynamical system.
10
SPEAKER_00
I think it just doesn t look good, honestly.
10
SPEAKER_01
Yeah, so that explosion could be destructive or constructive, and the probabilities are non-zero in both ends of the spectrum.
10
SPEAKER_00
I do feel like I have to try to be optimistic and so on.
10
SPEAKER_00
I think even in this case, I still am predominantly optimistic, but there s definitely
10
SPEAKER_01
Me too.
10
SPEAKER_01
Do you think we ll become a multi-planetary species?
10
SPEAKER_00
Probably yes, but I don t know if it s dominant feature of future humanity.
10
SPEAKER_00
There might be some people on some planets and so on, but I m not sure if it s like, yeah, if it s like a major player in our culture and so on.
10
SPEAKER_01
We still have to solve the drivers of self-destruction here on Earth.
10
SPEAKER_01
So just having a backup on Mars is not gonna solve the problem.
10
SPEAKER_00
So by the way, I love the backup on Mars.
10
SPEAKER_00
I think that s amazing.
10
SPEAKER_00
You should absolutely do that.
10
SPEAKER_00
And I m so thankful.
10
SPEAKER_00
Would you go to Mars?
10
SPEAKER_00
Personally, no.
10
SPEAKER_00
I do like Earth quite a lot.
10
SPEAKER_00
OK.
10
SPEAKER_00
I ll go to Mars.
10
SPEAKER_00
I ll go for you.
10
SPEAKER_00
I ll tweet at you from there.
10
SPEAKER_00
Maybe eventually I would, once it s safe enough.
10
SPEAKER_00
But I don t actually know if it s on my lifetime scale, unless I can extend it by a lot.
10
SPEAKER_00
I do think that, for example, a lot of people might disappear into virtual realities and stuff like that.
10
SPEAKER_00
I think that could be the major thrust of the cultural development of humanity, if it survives.
10
SPEAKER_00
It s just really hard to work in physical realm and go out there.
10
SPEAKER_00
I think, ultimately, all your experiences are in your brain.
10
SPEAKER_00
It s much easier to disappear into digital realm.
10
SPEAKER_00
I think people will find them more compelling, easier, safer.
10
SPEAKER_01
more interesting.
10
SPEAKER_01
So you re a little bit captivated by virtual reality, by the possible worlds, whether it s the metaverse or some other manifestation of that.
10
SPEAKER_00
Yeah.
11
SPEAKER_00
Yeah, it s really interesting.
11
SPEAKER_00
I m interested, just talking a lot to Carmack, where s the thing that s currently preventing that?
11
SPEAKER_01
Yeah.
11
SPEAKER_01
I mean, to be clear, I think what s interesting about the future is it s not that... I kind of feel like
11
SPEAKER_01
the variance in the human condition grows.
11
SPEAKER_01
That s the primary thing that s changing.
11
SPEAKER_01
It s not as much the mean of the distribution, it s like the variance of it.
11
SPEAKER_01
So there will probably be people on Mars, and there will be people in VR, and there will be people here on Earth.
11
SPEAKER_01
It s just like there will be so many more ways of being.
11
SPEAKER_01
And so I kind of feel like I see it as like a spreading out of a human experience.
11
SPEAKER_00
There s something about the internet that allows you to discover those little groups, and you gravitate to them.
11
SPEAKER_00
Something about your biology likes that kind of world, and that you find each other.
11
SPEAKER_01
And we ll have transhumanists, and then we ll have the Amish, and everything is just gonna coexist.
11
SPEAKER_00
The cool thing about it, because I ve interacted with a bunch of internet communities,
11
SPEAKER_00
is they don t know about each other.
11
SPEAKER_00
Like you can have a very happy existence, just like having a very close-knit community and not knowing about each other.
11
SPEAKER_00
You even sense this just having traveled to Ukraine.
11
SPEAKER_00
They don t know so many things about America.
11
SPEAKER_00
When you travel across the world, I think you experience this too.
11
SPEAKER_00
There are certain cultures that are like, they have their own thing going on.
11
SPEAKER_00
And so you can see that happening more and more and more and more in the future.
11
SPEAKER_00
We have little communities.
11
SPEAKER_01
Yeah.
11
SPEAKER_01
Yeah, I think so.
11
SPEAKER_01
That seems to be how it s going right now.
11
SPEAKER_01
And I don t see that trend like really reversing.
11
SPEAKER_01
I think people are diverse, and they re able to choose their own path in existence.
11
SPEAKER_01
And I sort of like celebrate that.
11
SPEAKER_00
And so... Will you spend so much time in the metaverse, in the virtual reality?
11
SPEAKER_00
Or which community are you?
11
SPEAKER_00
Are you the physicalist, the physical reality enjoyer, or do you see drawing a lot of pleasure and fulfillment in the digital world?
11
SPEAKER_01
Yeah, I think currently the virtual reality is not that compelling.
11
SPEAKER_01
I do think it can improve a lot, but I don t really know to what extent.
11
SPEAKER_01
Maybe there s actually even more exotic things you can think about with neural links or stuff like that.
11
SPEAKER_01
Currently, I kind of see myself as mostly a team human person.
11
SPEAKER_01
I love nature.
11
SPEAKER_01
I love harmony.
11
SPEAKER_01
I love people.
11
SPEAKER_01
I love humanity.
11
SPEAKER_01
I love emotions of humanity.
11
SPEAKER_01
And I just want to be like in this like solar punk little utopia.
11
SPEAKER_01
That s my happy place.
11
SPEAKER_01
My happy place is like people I love thinking about cool problems surrounded by lush, beautiful, dynamic nature.
11
SPEAKER_01
and secretly high tech in places that count.
11
SPEAKER_00
Places that count.
11
SPEAKER_00
So you use technology to empower that love for other humans and nature.
11
SPEAKER_01
Yeah, I think technology used like very sparingly.
11
SPEAKER_01
I don t love when it sort of gets in the way of humanity in many ways.
11
SPEAKER_01
I like just people being humans in a way.
11
SPEAKER_01
We sort of like slightly evolved and prefer, I think, just by default.
11
SPEAKER_00
People kept asking me, because they know you love reading.
11
SPEAKER_00
Are there particular books that you enjoyed that had an impact on you for silly or for profound reasons that you would recommend?
11
SPEAKER_00
You mentioned the vital question.
11
SPEAKER_01
many of course.
11
SPEAKER_01
I think in biology as an example, the vital question is a good one.
11
SPEAKER_01
Anything by Nick Lane, really life ascending, I would say is like a bit more potentially representative as like a summary of a lot of the things he s been talking about.
11
SPEAKER_01
I was very impacted by The Selfish Gene.
11
SPEAKER_01
I thought that was a really good book that helped me understand altruism as an example and where it comes from and just realizing that, you know, the selection is on the level of genes was a huge insight for me at the time.
11
SPEAKER_01
And it sort of like cleared up a lot of things for me.
11
SPEAKER_01
What do you think about
11
SPEAKER_00
the idea that ideas are the organisms, the memes.
11
SPEAKER_00
Yeah, love it, 100%.
11
SPEAKER_00
Are you able to walk around with that notion for a while, that there is an evolutionary kind of process with ideas as well?
11
SPEAKER_01
There absolutely is.
11
SPEAKER_01
There s memes, just like genes, and they compete, and they live in our brains.
11
SPEAKER_01
It s beautiful.
11
SPEAKER_00
Are we silly humans thinking that we re the organisms?
11
SPEAKER_00
Is it possible that the primary organisms are the ideas?
11
SPEAKER_01
Yeah, I would say like the ideas kind of live in the software of like our civilization in the minds and so on.
11
SPEAKER_00
We think as humans that the hardware is the fundamental thing.
11
SPEAKER_00
I, human, is a hardware entity, but it could be the software, right?
11
SPEAKER_01
Yeah, I would say like there needs to be some grounding at some point to like a physical reality.
11
SPEAKER_00
Yeah, but if we clone an Andre,
11
SPEAKER_00
the software is the thing, like, is the thing that makes that thing special, right?
11
SPEAKER_01
Yeah, I guess you re right.
11
SPEAKER_00
But then cloning might be exceptionally difficult.
11
SPEAKER_00
Like, there might be a deep integration between the software and the hardware in ways we don t quite understand.
11
SPEAKER_01
Well, from the alternate point of view, like, what makes me special is more like the gang of genes that are riding in my chromosomes, I suppose, right?
11
SPEAKER_01
Like, they re the replicating unit, I suppose.
11
SPEAKER_00
No, but that s just the compute.
11
SPEAKER_00
The thing that makes you special, sure, well,
11
SPEAKER_00
The reality is what makes you special is your ability to survive based on the software that runs on the hardware that was built by the genes.
11
SPEAKER_00
So the software is the thing that makes you survive, not the hardware.
11
SPEAKER_01
It s a little bit of both.
11
SPEAKER_01
It s just like a second layer.
11
SPEAKER_01
It s a new second layer that hasn t been there before the brain.
11
SPEAKER_01
They both coexist.
11
SPEAKER_01
But there s also layers of the software.
11
SPEAKER_00
I mean, it s an abstraction on top of abstractions.
11
SPEAKER_00
Selfish gene.
11
SPEAKER_01
Nick Lane, I would say sometimes books are like not sufficient.
11
SPEAKER_01
I like to reach for textbooks sometimes.
11
SPEAKER_01
I kind of feel like books are for too much of a general consumption sometimes, and they re too high up in the level of abstraction, and it s not good enough.
11
SPEAKER_01
So I like textbooks.
11
SPEAKER_01
I like The Cell.
11
SPEAKER_01
I think The Cell was pretty cool.
11
SPEAKER_01
That s why also I like the writing of Nick Lane, is because he s pretty willing to step one level down, and he doesn t, yeah, he s sort of, he s willing to go there.
11
SPEAKER_01
But he s also willing to sort of be throughout the stack, so he ll go down to a lot of detail, but then he will come back up, and I think he has a, yeah, basically I really appreciate that.
11
SPEAKER_00
That s why I love college, early college, even high school.
11
SPEAKER_00
Just textbooks on the basics.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
of computer science, of mathematics, of biology, of chemistry.
11
SPEAKER_00
Those are, they condense down.
11
SPEAKER_00
It s sufficiently general that you can understand both the philosophy and the details, but also you get homework problems and you get to play with it as much as you would if you weren t.
11
SPEAKER_00
programming stuff.
11
SPEAKER_01
Yeah.
11
SPEAKER_01
And then I m also suspicious of textbooks, honestly, because as an example, in deep learning, there s no like amazing textbooks and the field is changing very quickly.
11
SPEAKER_01
I imagine the same is true and say, synthetic biology and so on.
11
SPEAKER_01
These books like The Cell are kind of outdated.
11
SPEAKER_01
They re still high level.
11
SPEAKER_01
Like what is the actual real source of truth?
11
SPEAKER_01
It s people in wet labs working with cells, you know, sequencing genomes and
11
SPEAKER_01
Yeah, actually working with it.
11
SPEAKER_01
And I don t have that much exposure to that or what that looks like.
11
SPEAKER_01
So I still don t fully, I m reading through the cell and it s kind of interesting and I m learning, but it s still not sufficient, I would say, in terms of understanding.
11
SPEAKER_00
Well, it s a clean summarization of the mainstream narrative.
11
SPEAKER_00
But you have to learn that before you break out towards the cutting edge.
11
SPEAKER_00
Yeah.
11
SPEAKER_01
But what is the actual process of working with these cells and growing them and incubating them?
11
SPEAKER_01
And, you know, it s kind of like a massive cooking recipe.
11
SPEAKER_01
So making sure your cells lives and proliferate, and then you re sequencing them, running experiments, and just how that works, I think is kind of like the source of truth of, at the end of the day, what s really useful in terms of creating therapies and so on.
11
SPEAKER_00
Yeah, I wonder what, in the future, AI textbooks will be.
11
SPEAKER_00
Because, you know, there s artificial intelligence, the modern approach.
11
SPEAKER_00
I actually haven t read, if it s come out, the recent version, the recent, there s been a recent edition.
11
SPEAKER_00
I also saw there s a science and deep learning book.
11
SPEAKER_00
I m waiting for textbooks that are worth recommending, worth reading.
11
SPEAKER_00
It s tricky, because it s like papers.
11
SPEAKER_00
And code, code, code.
11
SPEAKER_01
Honestly, I find papers are quite good.
11
SPEAKER_01
I especially like the appendix of any paper as well.
11
SPEAKER_01
It s like the most detail you can have.
11
SPEAKER_00
It doesn t have to be cohesive, connected to anything else.
11
SPEAKER_00
You re just describing a very specific way you saw the particular thing.
11
SPEAKER_00
Yeah.
11
SPEAKER_01
Many times papers can be actually quite readable.
11
SPEAKER_01
Not always, but sometimes the introduction and the abstract is readable, even for someone outside of the field.
11
SPEAKER_01
This is not always true.
11
SPEAKER_01
And sometimes I think, unfortunately, scientists use complex terms, even when it s not necessary.
11
SPEAKER_01
I think that s harmful.
11
SPEAKER_00
I think there s no reason for that.
11
SPEAKER_00
And papers sometimes are longer than they need to be in the parts that...
11
SPEAKER_00
don t matter.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
Appendix should be long, but then the paper itself, you know, look at Einstein, make it simple.
11
SPEAKER_01
Yeah.
11
SPEAKER_01
But certainly I ve come across papers, I would say, say like synthetic biology or something that I thought were quite readable for the abstract and the introduction.
11
SPEAKER_01
And then you re reading the rest of it and you don t fully understand, but you kind of are getting a gist.
11
SPEAKER_01
And I think it s cool.
11
SPEAKER_00
What advice, you give advice to folks interested in machine learning and research, but in general, life advice to a young person, high school, early college, about how to have a career they can be proud of or a life they can be proud of?
11
SPEAKER_01
Yeah, I think I m very hesitant to give general advice.
11
SPEAKER_01
I think it s really hard.
11
SPEAKER_01
I ve mentioned like some of the stuff I ve mentioned is fairly general.
11
SPEAKER_01
I think like focus on just the amount of work you re spending on like a thing.
11
SPEAKER_01
Compare yourself only to yourself, not to others.
11
SPEAKER_01
That s good.
11
SPEAKER_01
I think those are fairly general.
11
SPEAKER_01
How do you pick the thing?
11
SPEAKER_01
you just have a deep interest in something, or try to find the argmax over the things that you re interested in.
11
SPEAKER_00
Argmax at that moment and stick with it.
11
SPEAKER_00
How do you not get distracted and switch to another thing?
11
SPEAKER_01
You can, if you like.
11
SPEAKER_00
Well, if you do an argmax repeatedly, every week, every month.
11
SPEAKER_00
It doesn t converge.
11
SPEAKER_00
It s a problem.
11
SPEAKER_01
Yeah, you can low-pass filter yourself in terms of what has consistently been true for you.
11
SPEAKER_01
But yeah, I definitely see how
11
SPEAKER_01
it can be hard, but I would say you re going to work the hardest on the thing that you care about the most.
11
SPEAKER_01
So low pass filter yourself and really introspect.
11
SPEAKER_01
In your past, what are the things that gave you energy and what are the things that took the energy away from you?
11
SPEAKER_01
Concrete examples.
11
SPEAKER_01
And usually from those concrete examples, sometimes patterns can emerge.
11
SPEAKER_01
I like it when things look like this when I m in these positions.
11
SPEAKER_00
So that s not necessarily the field, but the kind of stuff you re doing in a particular field.
11
SPEAKER_00
So for you, it seems like you were energized by implementing stuff, building actual things.
11
SPEAKER_01
Yeah, being low level, learning, and then also communicating so that others can go through the same realizations and shortening that gap.
11
SPEAKER_01
Because I usually have to do way too much work to understand a thing.
11
SPEAKER_01
And then I m like, okay, this is actually like, okay, I think I get it.
11
SPEAKER_01
And like, why was it so much work?
11
SPEAKER_01
It should have been much less work.
11
SPEAKER_01
And that gives me a lot of frustration.
11
SPEAKER_01
And that s why I sometimes go teach.
11
SPEAKER_00
So aside from the teaching you re doing now, putting out videos, aside from a potential Godfather part two, with the AGI at Tesla and beyond, what does the future for Anjay Karpathy hold?
11
SPEAKER_00
Have you figured that out yet or no?
11
SPEAKER_00
I mean, as you see through the fog of war that is all of our future, do you start seeing silhouettes of what that possible future could look like?
11
SPEAKER_01
The consistent thing I ve been always interested in, for me at least, is AI.
11
SPEAKER_01
That s probably what I m spending the rest of my life on, because I just care about it a lot.
11
SPEAKER_01
And I actually care about many other problems as well, like, say, aging, which I basically view as disease.
11
SPEAKER_01
And I care about that as well, but I don t think it s a good idea to go after it specifically.
11
SPEAKER_01
I don t actually think that humans will be able to come up with the answer.
11
SPEAKER_01
I think the correct thing to do is to ignore those problems, and you solve AI, and then use that to solve everything else.
11
SPEAKER_01
And I think there s a chance that this will work.
11
SPEAKER_01
I think it s a very high chance.
11
SPEAKER_01
And that s kind of like the way I m betting, at least.
11
SPEAKER_00
So when you think about AI, are you interested in all kinds of applications, all kinds of domains, and any domain you focus on will allow you to get insights to the big problem of AGI?
11
SPEAKER_01
Yeah, for me, it s the ultimate meta problem.
11
SPEAKER_01
I don t want to work on any one specific problem.
11
SPEAKER_01
There s too many problems.
11
SPEAKER_01
So how can you work on all problems simultaneously?
11
SPEAKER_01
You solve the meta problem, which to me is just intelligence.
11
SPEAKER_01
And how do you automate it?
11
SPEAKER_00
Is there cool small projects like Archive Sanity and so on that you re thinking about that the ML world can anticipate?
11
SPEAKER_01
There s always some fun side projects.
11
SPEAKER_01
Archive Sanity is one.
11
SPEAKER_01
Basically, there s way too many archive papers.
11
SPEAKER_01
How can I organize it and recommend papers and so on?
11
SPEAKER_01
I transcribed all of your podcasts.
11
SPEAKER_00
What did you learn from that experience, from transcribing the process of, you like consuming audio books and podcasts and so on.
11
SPEAKER_00
Here s a process that achieves closer to human level performance on annotation.
11
SPEAKER_01
Yeah.
11
SPEAKER_01
Well, I definitely was surprised that transcription with opening eyes whisper was working so well compared to what I m familiar with from Siri and a few other systems, I guess.
11
SPEAKER_01
It works so well.
11
SPEAKER_01
And that s what gave me some energy to try it out.
11
SPEAKER_01
And I thought it could be fun to run on podcasts.
11
SPEAKER_01
It s kind of not obvious to me why
11
SPEAKER_01
Whisper is so much better compared to anything else because I feel like there should be a lot of incentive for a lot of companies to produce transcription systems and that they ve done so over a long time.
11
SPEAKER_01
Whisper is not a super exotic model.
11
SPEAKER_01
It s a transformer.
11
SPEAKER_01
It takes male spectrograms and just outputs tokens of text.
11
SPEAKER_01
It s not crazy.
11
SPEAKER_01
The model and everything has been around for a long time.
11
SPEAKER_01
I m not actually 100% sure why this came up.
11
SPEAKER_00
It s not obvious to me either.
11
SPEAKER_00
It makes me feel like I m missing something.
11
SPEAKER_00
I m missing something.
11
SPEAKER_00
Yeah, because there is a huge, even Google and so on, YouTube transcription.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
Yeah, it s unclear.
11
SPEAKER_00
But some of it is also integrating into a bigger system.
11
SPEAKER_01
Yeah.
11
SPEAKER_00
So the user interface, how it s deployed and all that kind of stuff, maybe running it as an independent thing is much easier, like an order of magnitude easier than deploying to a large integrated system like YouTube transcription or anything like meetings, like Zoom has transcription, that s kind of crappy.
11
SPEAKER_00
But creating an interface where it detects the different individual speakers,
11
SPEAKER_00
it s able to display it in compelling ways, run it in real time, all that kind of stuff.
11
SPEAKER_00
Maybe that s difficult.
11
SPEAKER_00
That s the only explanation I have, because I m currently paying quite a bit for human transcription, human captions, annotation, and it seems like there s a huge incentive to automate that.
11
SPEAKER_00
It s very confusing.
11
SPEAKER_01
I don t know if you looked at some of the Whisper transcripts,
11
SPEAKER_01
Quite good.
11
SPEAKER_01
They re good.
11
SPEAKER_00
And especially in tricky cases.
11
SPEAKER_00
I ve seen Whisper s performance on like super tricky cases and it does incredibly well.
11
SPEAKER_00
So I don t know.
11
SPEAKER_00
A podcast is pretty simple.
11
SPEAKER_00
It s like high quality audio and you re speaking usually pretty clearly.
11
SPEAKER_00
And so I don t know.
11
SPEAKER_00
I don t know what OpenAI s plans are either.
11
SPEAKER_01
But yeah, there s always fun projects, basically.
11
SPEAKER_01
And stable diffusion also is opening up a huge amount of experimentation, I would say, in the visual realm and generating images and videos and movies, ultimately.
11
SPEAKER_00
Yeah, videos now.
11
SPEAKER_01
And so that s going to be pretty crazy.
11
SPEAKER_01
That s going to almost certainly work.
11
SPEAKER_01
And it s going to be really interesting when the cost of content creation is going to fall to zero.
11
SPEAKER_01
You used to need a painter for a few months to paint a thing.
11
SPEAKER_01
And now it s going to be speak to your phone to get your video.
11
SPEAKER_00
So Hollywood will start using that to generate scenes.
11
SPEAKER_00
which completely opens up, yeah, so you can make a movie like Avatar eventually for under a million dollars.
11
SPEAKER_01
Much less, maybe just by talking to your phone.
11
SPEAKER_01
I mean, I know it sounds kind of crazy.
11
SPEAKER_00
And then there d be some voting mechanism.
11
SPEAKER_00
Would there be a show on Netflix that s generated completely automatically?
11
SPEAKER_01
Semi-automatically, yeah.
11
SPEAKER_01
And what does it look like also when you can just generate it on demand and there s infinity of it?
11
SPEAKER_00
Yeah.
11
SPEAKER_00
Oh man.
11
SPEAKER_00
All the synthetic content.
11
SPEAKER_00
I mean, it s humbling because we treat ourselves as special for being able to generate art and ideas and all that kind of stuff.
11
SPEAKER_00
If that can be done in an automated way by AI.
11
SPEAKER_01
I think it s fascinating to me how the predictions of AI and what it s going to look like and what it s going to be capable of are completely inverted and wrong.
11
SPEAKER_01
And sci-fi of 50s and 60s was just like totally not right.
11
SPEAKER_01
They imagined AI as like super calculating theorem provers.
11
SPEAKER_01
And we re getting things that can talk to you about emotions.
11
SPEAKER_01
They can do art.
11
SPEAKER_01
It s just like weird.
11
SPEAKER_00
Are you excited about that future?
11
SPEAKER_00
Just AI as like hybrid systems.
11
SPEAKER_00
heterogeneous systems of humans and AIs talking about emotions, Netflix and chill with an AI system, legit, where the Netflix thing you watch is also generated by AI.
11
SPEAKER_01
I think it s going to be interesting for sure.
11
SPEAKER_01
And I think I m cautiously optimistic, but it s not obvious.
11
SPEAKER_00
Well, the sad thing is your brain and mine developed in a time where before
12
SPEAKER_00
or Twitter before the internet.
12
SPEAKER_00
So I wonder, people that are born inside of it might have a different experience.
12
SPEAKER_00
Like I, and maybe you, will still resist it.
12
SPEAKER_00
And the people born now will not.
12
SPEAKER_01
Well, I do feel like humans are extremely malleable.
12
SPEAKER_01
Yeah.
12
SPEAKER_01
And you re probably right.
12
SPEAKER_00
What is the meaning of life, Andre?
12
SPEAKER_00
We talked about sort of,
12
SPEAKER_00
the universe having a conversation with us humans or with the systems we create to try to answer, for the creator of the universe to notice us, we re trying to create systems that are loud enough to answer back.
12
SPEAKER_01
I don t know if that s the meaning of life.
12
SPEAKER_01
That s the meaning of life for some people.
12
SPEAKER_01
The first level answer I would say is anyone can choose their own meaning of life because we are a conscious entity and it s beautiful, number one.
12
SPEAKER_01
But I do think that a deeper meaning of life, if someone is interested, is along the lines of what the hell is all this and why?
12
SPEAKER_01
If you look into fundamental physics and the quantum field theory and the standard model, they re very complicated.
12
SPEAKER_01
There s this 19 free parameters of our universe.
12
SPEAKER_01
And what s going on with all this stuff?
12
SPEAKER_01
And why is it here?
12
SPEAKER_01
And can I hack it?
12
SPEAKER_01
Can I work with it?
12
SPEAKER_01
Is there a message for me?
12
SPEAKER_01
Am I supposed to create a message?
12
SPEAKER_01
And so I think there s some fundamental answers there.
12
SPEAKER_01
But I think there s actually even like, you can t actually really make dent in those without more time.
12
SPEAKER_01
And so to me also, there s a big question around just getting more time, honestly.
12
SPEAKER_01
Yeah, that s kind of like what I think about quite a bit as well.
12
SPEAKER_00
So kind of the ultimate, or at least first way to sneak up to the why question is to try to escape
12
SPEAKER_00
the system, the universe.
12
SPEAKER_00
And then for that, you sort of backtrack and say, okay, for that, it s gonna take a very long time.
12
SPEAKER_00
So the why question boils down from an engineering perspective to how do we extend?
12
SPEAKER_01
Yeah, I think that s the question number one, practically speaking, because you can t, you re not gonna calculate the answer to the deeper questions in time you have.
12
SPEAKER_00
And that could be extending your own lifetime or extending just the lifetime of human civilization.
12
SPEAKER_01
of whoever wants to.
12
SPEAKER_01
Not many people might not want that.
12
SPEAKER_01
But I think people who do want that, I think it s probably possible.
12
SPEAKER_01
And I don t know that people fully realize this.
12
SPEAKER_01
I kind of feel like people think of death as an inevitability.
12
SPEAKER_01
But at the end of the day, this is a physical system.
12
SPEAKER_01
Some things go wrong.
12
SPEAKER_01
It makes sense why things like this happen, evolutionarily speaking.
12
SPEAKER_01
And there s most certainly interventions that mitigate it.
12
SPEAKER_00
That d be interesting if death is eventually looked at as a fascinating thing that used to happen to humans.
12
SPEAKER_01
I don t think it s unlikely.
12
SPEAKER_01
I think it s likely.
12
SPEAKER_00
And it s up to our imagination to try to predict what the world without death looks like.
12
SPEAKER_00
I think the values will completely change.
12
SPEAKER_01
Could be.
12
SPEAKER_01
I don t really buy all these ideas that, oh, without death there s no meaning, there s nothing.
12
SPEAKER_01
I don t intuitively buy all those arguments.
12
SPEAKER_01
I think there s plenty of meaning, plenty of things to learn.
12
SPEAKER_01
They re interesting, exciting, I want to know, I want to calculate.
12
SPEAKER_01
I want to improve the condition of all the humans and organisms that are alive.
12
SPEAKER_00
Yeah, the way we find meaning might change.
12
SPEAKER_00
There is a lot of humans, probably including myself, that finds meaning in the finiteness of things.
12
SPEAKER_00
But that doesn t mean that s the only source of meaning.
12
SPEAKER_01
Yeah.
12
SPEAKER_01
I do think many people will go with that, which I think is great.
12
SPEAKER_01
I love the idea that people can just choose their own adventure.
12
SPEAKER_01
Like you are born as a conscious free entity by default, I d like to think.
12
SPEAKER_01
And you have your unalienable rights for life.
12
SPEAKER_00
In the pursuit of happiness.
12
SPEAKER_00
I don t know if you have that.
12
SPEAKER_00
and the nature, the landscape of happiness.
12
SPEAKER_01
And you can choose your own adventure mostly, and that s not fully true, but.
12
SPEAKER_00
I still am pretty sure I m an NPC, but an NPC can t know it s an NPC.
12
SPEAKER_00
There could be different degrees and levels of consciousness.
12
SPEAKER_00
I don t think there s a more beautiful way to end it.
12
SPEAKER_00
Andre, you re an incredible person.
12
SPEAKER_00
I m really honored you would talk with me.
12
SPEAKER_00
Everything you ve done for the machine learning world, for the AI world,
12
SPEAKER_00
to just inspire people, to educate millions of people.
12
SPEAKER_00
It s been great and I can t wait to see what you do next.
12
SPEAKER_00
It s been an honor, man.
12
SPEAKER_00
Thank you so much for talking today.
12
SPEAKER_01
Awesome, thank you.
12
SPEAKER_00
Thanks for listening to this conversation with Andrej Karpathy.
12
SPEAKER_00
To support this podcast, please check out our sponsors in the description.
12
SPEAKER_00
And now, let me leave you with some words from Samuel Carlin.
12
SPEAKER_00
The purpose of models is not to fit the data, but to sharpen the questions.
12
SPEAKER_00
Thanks for listening and hope to see you next time.