0
SPEAKER_02
The following is a conversation with Sam Altman, his second time in the podcast.
0
SPEAKER_02
He is the CEO of OpenAI, the company behind GPT-4, Chad GPT, Sora, and perhaps one day, the very company that will build AGI.
0
SPEAKER_02
And now, a quick few second mention of each sponsor.
0
SPEAKER_02
Check them out in the description.
0
SPEAKER_02
It s the best way to support this podcast.
0
SPEAKER_02
We got a new sponsor, Cloaked, for protecting your personal information, Shopify for selling stuff online, BetterHelp for helping out your mind, and ExpressVPN for protecting your privacy.
0
SPEAKER_02
your privacy and security on the interwebs.
0
SPEAKER_02
Choose wisely, my friends.
0
SPEAKER_02
Also, if you want to work with our amazing team we re always hiring, or if you just want to get in touch with me, go to lexfriedman.com slash contact.
0
SPEAKER_02
And now, onto the full ad reads.
0
SPEAKER_02
As always, no ads in the middle.
0
SPEAKER_02
I try to make this interesting, but if you must skip them, friends, please do check out our sponsors.
0
SPEAKER_02
I enjoy their stuff.
0
SPEAKER_02
Maybe you will, too.
0
SPEAKER_02
This episode is brought to you by Cloaked, a sponsor I didn t know existed until quite recently and always thought a thing like this should exist and I couldn t quite find a thing like it that existed and once I found it, it was pretty awesome.
0
SPEAKER_02
It s a platform that lets you generate new email addresses and phone numbers every time you sign up for a website.
0
SPEAKER_02
So it s called a masked email, which basically creates, I guess you could say it s a fake email that hides your actual email.
0
SPEAKER_02
It s not fake in that it actually exists and persists throughout time, and the website thinks it s real, but it just forwards to your actual email.
0
SPEAKER_02
You can set up the forwarding.
0
SPEAKER_02
The point is, the website or service that you sign up for doesn t know your actual phone number.
0
SPEAKER_02
It doesn t know your actual email.
0
SPEAKER_02
So this is a really interesting idea, because when you sign up to different websites, there s a kind of contract, unspoken contract, that the email you provide and the phone number you provide will not be abused.
0
SPEAKER_02
For the kind of abuse I m talking about, in sort of the best case, just spammed, or in the worst case, that email or phone number being sold out there, and then you get not just spammed from one source, but spammed from all of the sources all over the place.
0
SPEAKER_02
Anyway, this is just a smart thing to protect yourself.
0
SPEAKER_02
And it also does basic password manager stuff.
0
SPEAKER_02
So you can think of Cloaked as a great password manager with extra privacy superpowers.
0
SPEAKER_02
You can go to cloaked.com slash Lex to get 14 days free or for a limited time use code LexPod when signing up to get 25% off an annual Cloaked plan.
0
SPEAKER_02
This episode is also brought to you by Shopify, a platform designed for anyone, yes, anyone including me, to sell anywhere with a great looking online store.
0
SPEAKER_02
I used it to sell some t-shirts at lexrumin.com slash store.
0
SPEAKER_02
You can check it out.
0
SPEAKER_02
I used the most basic store.
0
SPEAKER_02
It took just a few minutes and the store was up.
0
SPEAKER_02
from the shirt design being finished to the store being alive and being able to sell t-shirts and ship those t-shirts thanks to the integration with a third party which there s thousands of integrations with a third party so for t-shirts that s like on-demand printing so you don t have to take care of the shipping and the printing and all that kind of stuff
0
SPEAKER_02
All of that is integrated, super easy to do, and this works for any kind of business that sells stuff online.
0
SPEAKER_02
You can integrate into your own website, or you can sell it on Shopify itself, which is what I do.
0
SPEAKER_02
You can sign up for a $1 per month trial period at shopify.com slash lex, all lowercase, go to shopify.com slash lex to take your business to the next level today.
0
SPEAKER_02
This episode is also brought to you by BetterHelp, spelled H-E-L-P.
0
SPEAKER_02
Help, they figure out what you need and match you with a licensed therapist in under 48 hours.
0
SPEAKER_02
Works for individuals, works for couples.
0
SPEAKER_02
I m a huge fan of talking as a way of exploring the human mind.
0
SPEAKER_02
Two people talking with a motivation and a goal in mind.
0
SPEAKER_02
of surfacing certain kinds of problems and alleviating those kinds of problems.
0
SPEAKER_02
Sometimes the surfacing in itself does a lot of the alleviation.
0
SPEAKER_02
Returning to a time in the past when trauma happened and to reframe it in a way that helps you understand, that helps you forgive, that helps you let go.
0
SPEAKER_02
All of that, it s really powerful.
0
SPEAKER_02
And BetterHelp just is an accessible way of doing that, or at least trying talk therapy.
0
SPEAKER_02
So they ve helped a lot of people.
0
SPEAKER_02
4.4 million people got help.
0
SPEAKER_02
So you could be one of those.
0
SPEAKER_02
If you wanna try, check them out at betterhelp.com slash lex and save in your first month.
0
SPEAKER_02
That s betterhelp.com slash lex.
0
SPEAKER_02
This episode is also brought to you by ExpressVPN.
0
SPEAKER_02
I love that there s a kind of privacy theme to the sponsors in this episode.
0
SPEAKER_02
I think everybody should be using a VPN for many reasons.
0
SPEAKER_02
One, it can allow you to geographically
0
SPEAKER_02
Transport yourself.
0
SPEAKER_02
But the main reason is it just adds this extra layer of security and privacy between you and the ISP.
0
SPEAKER_02
That they say you re technically not supposed to be collecting the data when you use things like Chroma Cognito, but they can be collecting the data.
0
SPEAKER_02
I don t know how the laws of that works.
0
SPEAKER_02
but I wouldn t trust it, so a VPN is essential.
0
SPEAKER_02
For that, my favorite VPN for many, many, many, many, many, many years has been ExpressVPN.
0
SPEAKER_02
Big sexy button, still works.
0
SPEAKER_02
Looks different, but still works on any operating system.
0
SPEAKER_02
My favorite being Linux.
0
SPEAKER_02
I can talk forever about why I love Linux.
0
SPEAKER_02
I wonder if Linux will be around with all this AI, with all this rapid AI development.
0
SPEAKER_02
Maybe programmers, programming as a way of life, as a recreation for millions, as a profession for millions, will die out and there ll only be a handful, a few, like the cobalt programmers of today that carry the flag of knowing what Linux is, how to spell Linux, let alone use it, I wonder.
0
SPEAKER_02
Hopefully not, because there s always room for optimizing at every level, the compilation from the human language to the AI language to the machine language to the zeros and ones, the compilation of the entire stack.
0
SPEAKER_02
I think there s a lot of jobs to be had, a lot of really,
0
SPEAKER_02
Profitable, well-paying jobs to be had there, but maybe not millions of people are needed.
0
SPEAKER_02
Maybe there ll be millions of people that program with just natural language, with just words, English or whatever new language we create, that the whole world can use.
0
SPEAKER_02
And the whole world, in using, can help break down the barriers of language.
0
SPEAKER_02
We arrived here, friends, when we started at the meager explanation of the use of a VPN.
0
SPEAKER_02
You can also take this journey by going to expressvpn.com slash LexPod for an extra three months free.
0
SPEAKER_02
This is a Lex Friedman podcast.
0
SPEAKER_02
To support it, please check out our sponsors in the description.
0
SPEAKER_02
And now, dear friends, here s Sam Altman.
0
SPEAKER_02
Take me through the OpenAI board saga that started on Thursday, November 16th, maybe Friday, November 17th for you.
0
SPEAKER_00
That was definitely the most painful professional experience of my life and chaotic and shameful and upsetting and a bunch of other negative things.
0
SPEAKER_01
There were great things about it too and I wish it had not been
0
SPEAKER_01
in such an adrenaline rush that I wasn t able to stop and appreciate them at the time.
0
SPEAKER_01
I came across this old tweet of mine, or this tweet of mine from that time period, which was like going to your own eulogy, watching people say all these great things about you, and just unbelievable support from people I love and care about.
0
SPEAKER_01
That was really nice.
0
SPEAKER_00
That whole weekend, I kind of felt, with one big exception, I felt a great deal of love and very little hate.
0
SPEAKER_00
even though it felt like I have no idea what s happening and what s going to happen here, and this feels really bad.
0
SPEAKER_01
There were definitely times I thought it was going to be one of the worst things to ever happen for AI safety.
0
SPEAKER_01
I also think I m happy that it happened relatively early.
0
SPEAKER_01
I thought at some point between when OpenAI started and when we created AGI,
0
SPEAKER_00
There was going to be something crazy and explosive that happened, but there may be more crazy and explosive things still to happen.
0
SPEAKER_00
It still, I think, helped us build up some resilience and be ready for
0
SPEAKER_00
more challenges in the future.
0
SPEAKER_02
But the thing you had a sense that you would experience is some kind of power struggle.
0
SPEAKER_01
The road to AGI should be a giant power struggle, like the world should.
0
SPEAKER_01
Well, not should, I expect that to be the case.
0
SPEAKER_02
And so you have to go through that, like you said, iterate as often as possible in figuring out how to have a board structure, how to have organization, how to have the kind of people that you re working with, how to communicate all that in order to deescalate the power struggle as much as possible, pacify it.
0
SPEAKER_01
But at this point it feels,
0
SPEAKER_01
you know, like something that was in the past that was really unpleasant and really difficult and painful, but we re back to work and things are so busy and so intense that I don t spend a lot of time thinking about it.
0
SPEAKER_01
There was a time after, there was like this fugue state for kind of like the month after, maybe 45 days after.
0
SPEAKER_01
that was, I was just sort of like drifting through the days.
0
SPEAKER_01
I was so out of it.
0
SPEAKER_01
I was feeling so down.
0
SPEAKER_01
Just at a personal psychological level.
0
SPEAKER_01
Yeah.
0
SPEAKER_01
Really painful.
0
SPEAKER_01
And hard to like have to keep running open AI in the middle of that.
0
SPEAKER_01
I just wanted to like crawl into a cave and kind of recover for a while.
0
SPEAKER_01
But, you know, now it s like we re just back to working on the mission.
0
SPEAKER_02
Well, it s still useful to go back there and reflect on board structures, on
0
SPEAKER_02
power dynamics on how companies are run, the tension between research and product development and money and all this kind of stuff, so that you, who have a very high potential of building AGI, would do so in a slightly more organized, less dramatic way in the future.
0
SPEAKER_02
So there s value there to go, both the personal psychological aspects of you as a leader and also just the board structure and all this kind of
0
SPEAKER_02
messy stuff.
0
SPEAKER_01
Definitely learned a lot about structure and incentives and what we need out of a board.
0
SPEAKER_01
I think it is valuable that this happened now in some sense.
0
SPEAKER_01
I think this is probably not like the last high-stress moment of OpenAI, but it was quite a high-stress moment.
0
SPEAKER_01
My company very nearly got destroyed.
0
SPEAKER_01
We think a lot about
0
SPEAKER_01
many of the other things we ve got to get right for AGI, but thinking about how to build a resilient org and how to build a structure that will stand up to a lot of pressure in the world, which I expect more and more as we get closer, I think that s super important.
0
SPEAKER_02
Do you have a sense of how deep and rigorous the deliberation process by the board was?
0
SPEAKER_02
Can you shine some light
0
SPEAKER_02
On just human dynamics involved in situations like this.
0
SPEAKER_02
Was it just a few conversations and all sudden it escalates and Why don t we fire Sam kind of thing?
0
SPEAKER_00
I think I Think the board members were are well-meaning people on the whole And I believe that In stressful situations
0
SPEAKER_00
where people feel time pressure or whatever.
0
SPEAKER_00
People understandably make suboptimal decisions, and I think one of the challenges for
0
SPEAKER_01
OpenAI will be, we re gonna have to have a board and a team that are good at operating under pressure.
0
SPEAKER_01
Do you think the board had too much power?
0
SPEAKER_01
I think boards are supposed to have a lot of power.
0
SPEAKER_01
But one of the things that we did see is in most corporate structures, boards are usually answerable to shareholders.
0
SPEAKER_01
Sometimes people have like super voting shares or whatever.
0
SPEAKER_01
In this case, and I think one of the things with our structure that we maybe should have thought about more than we did, is that the board of a nonprofit has
0
SPEAKER_01
unless you put other rules in place, like quite a lot of power.
0
SPEAKER_01
They don t really answer to anyone but themselves, and there s ways in which that s good, but what we d really like is for the board of OpenAI to answer to the world as a whole as much as that s a practical thing.
0
SPEAKER_02
So there s a new board announced?
0
SPEAKER_01
Yeah.
0
SPEAKER_02
There s, I guess, a new smaller board at first, and now there s a new final board.
0
SPEAKER_02
Not a final board yet.
0
SPEAKER_02
We ve added some, we ll add more.
0
SPEAKER_02
Added some, okay.
0
SPEAKER_02
is fixed in the new one that was perhaps broken in the previous one.
0
SPEAKER_01
The old board sort of got smaller over the course of about a year.
0
SPEAKER_01
It was nine and then it went down to six.
0
SPEAKER_01
And then we couldn t agree on who to add.
0
SPEAKER_01
And the board also
0
SPEAKER_01
I think didn t have a lot of experienced board members and a lot of the new board members at OpenAI have just have more experience as board members.
0
SPEAKER_01
I think that ll help.
0
SPEAKER_02
It s been criticized, some of the people that are added to the board.
0
SPEAKER_02
I heard a lot of people criticizing the addition of Larry Summers, for example.
0
SPEAKER_02
What s the process of selecting the board like?
0
SPEAKER_02
What s involved in that?
0
SPEAKER_01
So, Brett and Larry were kind of decided in the heat of the moment over this very tense weekend.
0
SPEAKER_01
And that weekend was like a real roller coaster.
0
SPEAKER_01
It was like a lot of ups and downs.
0
SPEAKER_01
And we were trying to agree on
0
SPEAKER_01
new board members that both sort of the executive team here and the old board members felt would be reasonable.
0
SPEAKER_01
Larry was actually one of their suggestions, the old board members.
0
SPEAKER_01
Brett, I think, I had even previous to that weekend suggested, but he was busy and didn t want to do it and then we really needed help and would.
0
SPEAKER_01
We talked about a lot of other people too, but that was
0
SPEAKER_00
I felt like if I was going to come back, I needed new board members.
0
SPEAKER_01
I didn t think I could work with the old board again in the same configuration, although we then decided – and I m grateful that Adam would stay – but we considered various configurations, decided we wanted to get to a board of three.
0
SPEAKER_01
and had to find two new board members over the course of sort of a short period of time.
0
SPEAKER_01
Those were decided honestly without – That s like you kind of do that on the battlefield.
0
SPEAKER_01
You don t have time to design a rigorous process then.
0
SPEAKER_01
For new board members since,
0
SPEAKER_01
new board members will add going forward.
0
SPEAKER_01
We have some criteria that we think are important for the board to have, different expertise that we want the board to have.
0
SPEAKER_01
Unlike hiring an executive where you need them to do one role well, the board needs to do a whole role of kind of governance and thoughtfulness well.
0
SPEAKER_01
One thing that Brett says which I really like is that we want to hire board members in slates, not as
1
SPEAKER_01
individuals one at a time and thinking about a group of people that will bring
1
SPEAKER_01
non-profit expertise, expertise in running companies, sort of good legal and governance expertise.
1
SPEAKER_01
That s kind of what we ve tried to optimize for.
1
SPEAKER_02
So it s technical savvy important for the individual board members?
1
SPEAKER_01
Not for every board member, but for certainly some you need that.
1
SPEAKER_01
That s part of what the board needs to do.
1
SPEAKER_02
So, I mean, the interesting thing that people probably don t understand about OpenAI, I certainly don t, is like all the details of running the business.
1
SPEAKER_02
When they think about the board, given the drama, and think about you, they think about like,
1
SPEAKER_02
if you reach AGI or you reach some of these incredibly impactful products and you build them and deploy them, what s the conversation with the board like?
1
SPEAKER_02
And they kind of think, all right, what s the right squad to have in that kind of situation to deliberate?
1
SPEAKER_01
Look, I think you definitely need some technical experts there.
1
SPEAKER_01
And then you need some people who are like, how can we deploy this in a way that will help people in the world the most and people who have a very different perspective?
1
SPEAKER_01
You know, I think a mistake that you or I might make is to think that only the technical understanding matters.
1
SPEAKER_01
And that s definitely part of the conversation you want that board to have.
1
SPEAKER_01
But there s a lot more about how that s going to just like impact society and people s lives that you really want represented in there too.
1
SPEAKER_01
And you re just kind of, are you looking at the track record of people or you re just having conversations?
1
SPEAKER_01
Track record is a big deal.
1
SPEAKER_01
You of course have a lot of conversations, but I, um,
1
SPEAKER_01
There s some roles where I kind of totally ignore track record and just look at slope, kind of ignore the y-intercept.
1
SPEAKER_01
Thank you.
1
SPEAKER_01
Thank you for making it mathematical for the audience.
1
SPEAKER_01
For a board member, I do care much more about the y-intercept.
1
SPEAKER_01
I think there is something deep to say about track record there.
1
SPEAKER_01
and experience is sometimes very hard to replace.
1
SPEAKER_02
Do you try to fit a polynomial function or exponential one to the track record?
1
SPEAKER_02
That s not that, analogy doesn t carry that far.
1
SPEAKER_02
All right, you mentioned some of the low points that weekend.
1
SPEAKER_02
What were some of the low points psychologically for you?
1
SPEAKER_02
Did you consider going to the Amazon jungle and just taking ayahuasca and disappearing forever or?
1
SPEAKER_01
I mean, there s so many low, like it was very bad.
1
SPEAKER_01
period of time.
1
SPEAKER_01
There were great high points too.
1
SPEAKER_01
My phone was just sort of nonstop blowing up with nice messages from people I work with every day, people I hadn t talked to in a decade.
1
SPEAKER_01
I didn t get to appreciate that as much as I should have because I was just in the middle of this firefight, but that was really nice.
1
SPEAKER_00
But on the whole, it was a very painful weekend and also just a very
1
SPEAKER_01
It was like a battle fought in public to a surprising degree, and that was extremely exhausting to me, much more than I expected.
1
SPEAKER_01
I think fights are generally exhausting, but this one really was.
1
SPEAKER_00
Friday afternoon, I really couldn t get much in the way of answers, but I also was just like, well, the board gets to do this.
1
SPEAKER_01
I m going to think for a little bit about what I want to do, but I ll try to find the blessing in disguise here.
1
SPEAKER_01
I was like, well, I
1
SPEAKER_01
My current job at OpenAI is or it was to run a decently sized company at this point.
1
SPEAKER_01
The thing I d always liked the most was just getting to work with the researchers.
1
SPEAKER_01
I was like, yeah, I can just go do a very focused AGI research effort.
1
SPEAKER_01
And I got excited about that.
1
SPEAKER_01
It didn t even occur to me at the time to like possibly that this was all going to get undone.
1
SPEAKER_01
This was like Friday afternoon.
1
SPEAKER_01
So you ve accepted the death of this previous – Very quickly.
1
SPEAKER_01
Very quickly.
1
SPEAKER_01
I mean, I went through like a little period of confusion and rage, but very quickly.
1
SPEAKER_01
And by Friday night, I was like talking to people about what was going to be next.
1
SPEAKER_01
And I was excited about that.
1
SPEAKER_01
I think it was Friday night, evening for the first time that I heard from the exec team here, which is like, hey, we re going to like fight this and you know, we think whatever.
1
SPEAKER_01
And then I went to bed just still being like, okay, excited, like onward.
1
SPEAKER_01
Were you able to sleep?
1
SPEAKER_01
Not a lot.
1
SPEAKER_01
It was one of the weird things was this like period of four, four and a half days where
1
SPEAKER_01
sort of didn t sleep much, didn t eat much, and still kind of had like a surprising amount of energy.
1
SPEAKER_01
It was, you learn like a weird thing about adrenaline and wartime.
1
SPEAKER_02
So you kind of accepted the death of a, you know, this baby opening your eyes.
1
SPEAKER_01
And I was excited for the new thing.
1
SPEAKER_01
I was just like, okay, this was crazy, but whatever.
1
SPEAKER_02
It s a very good coping mechanism.
1
SPEAKER_01
And then Saturday morning, two of the board members called and said, hey, we, you know, destabilized, we didn t mean to destabilize things.
1
SPEAKER_01
We don t want to destroy a lot of value here.
1
SPEAKER_01
You know, can we talk about you coming back?
1
SPEAKER_01
I immediately didn t want to do that, but I thought a little more and I was like, well, I don t really care about the people here, the partners, shareholders, all of the – I love this company.
1
SPEAKER_01
So I thought about it and I was like, well, okay, but here s the stuff I would need.
1
SPEAKER_00
Then the most painful time of all was over the course of that weekend,
1
SPEAKER_01
I kept thinking and being told, and we all kept, not just me, the whole team here kept thinking while we were trying to keep OpenAI stabilized while the whole world was trying to break it apart, people trying to recruit, whatever.
1
SPEAKER_01
We kept being told, all right, we re almost done, we re almost done, we just need a little bit more time.
1
SPEAKER_01
It was this very confusing state.
1
SPEAKER_01
Then Sunday evening, when again, every few hours, I expected that we were going to be done and we re going to figure out a way for me to return.
1
SPEAKER_01
Things go back to how they were.
1
SPEAKER_01
The board then appointed a new interim CEO.
1
SPEAKER_01
And then I was like, I mean, that feels really bad.
1
SPEAKER_01
That was the low point of the whole thing.
1
SPEAKER_01
You know, I ll tell you something, it felt very painful, but I felt a lot of love that whole weekend.
1
SPEAKER_01
It was not, other than that one moment, Sunday night, I would not characterize my emotions as anger or hate, but I really just like, I felt a lot of love from people towards people.
1
SPEAKER_01
It was like painful, but it would like, the dominant emotion of the weekend was love, not hate.
1
SPEAKER_02
You ve spoken highly of Mira Moradi, that she helped especially as you put in a tweet, in the quiet moments when it counts.
1
SPEAKER_02
Perhaps we could take a bit of a tangent.
1
SPEAKER_02
What do you admire about Mira?
1
SPEAKER_01
Well, she did a great job during that weekend in a lot of chaos, but people often see leaders in the moment, in like the crisis moments, good or bad.
1
SPEAKER_01
But a thing I really value in leaders is how people act on a boring Tuesday at 9.46 in the morning.
1
SPEAKER_01
And in just sort of the normal drudgery of the day-to-day.
1
SPEAKER_01
How someone shows up in a meeting, the quality of the decisions they make.
1
SPEAKER_02
That was what I meant about the quiet moments.
1
SPEAKER_02
Meaning like most of the work is done on a day-by-day, in a meeting-by-meeting, just
1
SPEAKER_02
be present and make great decisions.
1
SPEAKER_01
Yeah, I mean, look, what you have wanted to spend the last 20 minutes about, and I understand, is like this one very dramatic weekend.
1
SPEAKER_02
Yeah.
1
SPEAKER_01
But that s not really what Opening Eye is about.
1
SPEAKER_01
Opening Eye is really about the other seven years.
1
SPEAKER_02
Well, yeah, human civilization is not about
1
SPEAKER_02
the invasion of the Soviet Union by Nazi Germany, but still that s something people focus on.
1
SPEAKER_02
It gives us an insight into human nature, the extremes of human nature, and perhaps some of the damage and some of the triumphs of human civilization can happen in those moments, so it s like illustrative.
1
SPEAKER_02
Let me ask you about Ilya.
1
SPEAKER_02
Is he being held hostage in a secret nuclear facility?
1
SPEAKER_02
No.
1
SPEAKER_02
What about a regular secret facility?
1
SPEAKER_02
No.
1
SPEAKER_02
What about a nuclear non-secret facility?
1
SPEAKER_02
Neither of that, not that either.
1
SPEAKER_02
I mean, this is becoming a meme at some point.
1
SPEAKER_02
You ve known Ilya for a long time.
1
SPEAKER_02
He was obviously in part of this drama with the board and all that kind of stuff.
1
SPEAKER_02
What s your relationship with him now?
1
SPEAKER_01
I love Ilya.
1
SPEAKER_01
I have tremendous respect for Ilya.
1
SPEAKER_01
I don t have anything I can say about his plans right now.
1
SPEAKER_01
That s a question for him.
1
SPEAKER_01
But I really hope we work together for certainly the rest of my career.
1
SPEAKER_01
He s a little bit younger than me.
1
SPEAKER_01
Maybe he works a little bit longer.
1
SPEAKER_02
There s a meme that he saw something.
1
SPEAKER_02
He maybe saw AGI and that gave him a lot of worry internally.
1
SPEAKER_02
What did Ilya see?
1
SPEAKER_01
Ilya has not seen AGI.
1
SPEAKER_01
None of us have seen AGI.
1
SPEAKER_01
We have not built AGI.
1
SPEAKER_01
I do think one of the many things that I really love about Ilya is he takes AGI and the safety concerns, broadly speaking, including things like the impact this is going to have on society, very seriously.
1
SPEAKER_01
As we continue to make significant progress, Ilya is one of the people that I ve spent the most time over the last couple of years talking about what this is going to mean, what we need to do to ensure we get it right, to ensure that we succeed at the mission.
1
SPEAKER_00
Ilya did not see AGI, but Ilya is a
1
SPEAKER_00
credit to humanity in terms of how much he thinks and worries about making sure we get this right.
1
SPEAKER_02
I ve had a bunch of conversations with him in the past.
1
SPEAKER_02
I think when he talks about technology, he s always doing this long-term thinking type of thing.
1
SPEAKER_02
So he s not thinking about what this is going to be in a year, he s thinking about it in 10 years.
1
SPEAKER_02
Just thinking from first principles like, okay,
1
SPEAKER_02
if the scales, what are the fundamentals here, where is this going?
1
SPEAKER_02
And so that s a foundation for them thinking about all the other safety concerns and all that kind of stuff, which makes him a really fascinating human to talk with.
1
SPEAKER_02
Do you have any idea why he s been kind of quiet?
1
SPEAKER_02
Is it he s just doing some soul searching?
1
SPEAKER_01
Again, I don t want to speak for Ilya.
1
SPEAKER_01
I think that you should ask him that.
1
SPEAKER_00
He s definitely a thoughtful guy.
1
SPEAKER_01
I think Avelio is always on a soul search in a really good way.
1
SPEAKER_02
Yes.
1
SPEAKER_02
Yeah.
1
SPEAKER_02
Also, he appreciates the power of silence.
1
SPEAKER_02
Also, I m told he can be a silly guy, which I ve never seen that side of him.
1
SPEAKER_02
It s very sweet when that happens.
1
SPEAKER_01
I ve never witnessed a silly Ilio, but I look forward to that as well.
1
SPEAKER_01
I was at a dinner party with him recently, and he was playing with a puppy.
1
SPEAKER_01
And he was in a very silly mood, very endearing.
1
SPEAKER_01
And I was thinking, oh man, this is not the side of the Ilio that the world sees the most.
1
SPEAKER_02
So just to wrap up this whole saga, are you feeling good about the board structure, about all of this and like where it s moving?
1
SPEAKER_01
I feel great about the new board.
1
SPEAKER_01
In terms of the structure of OpenAI, I, you know, one of the board s tasks is to look at that and see where we can make it more robust.
1
SPEAKER_01
We wanted to get new board members in place first.
1
SPEAKER_01
But, you know, we clearly learned a lesson about structure throughout this process.
1
SPEAKER_01
I don t have, I think, super deep things to say.
1
SPEAKER_01
It was a crazy, very painful experience.
1
SPEAKER_01
I think it was like a perfect storm of weirdness.
1
SPEAKER_01
It was like a preview for me of what s going to happen as the stakes get higher and higher and the need that we have like robust governance structures and processes and people.
1
SPEAKER_00
I am kind of happy it happened when it did.
1
SPEAKER_02
a shockingly painful thing to go through.
1
SPEAKER_02
Did it make you be more hesitant in trusting people?
1
SPEAKER_01
Yes.
1
SPEAKER_01
Just on a personal level?
1
SPEAKER_01
Yes.
1
SPEAKER_01
I think I m an extremely trusting person.
1
SPEAKER_01
I ve always had a life philosophy of, you know, like, don t worry about all of the paranoia, don t worry about the edge cases.
1
SPEAKER_01
You know, you get a little bit screwed in exchange for getting to live with your guard down.
1
SPEAKER_01
And this was so shocking to me, I was so caught off guard.
1
SPEAKER_01
that it has definitely changed.
1
SPEAKER_01
And I really don t like this.
1
SPEAKER_01
It s definitely changed how I think about just like default trust of people and planning for the bad scenarios.
1
SPEAKER_01
You gotta be careful with that.
1
SPEAKER_01
Are you worried about becoming a little too cynical?
1
SPEAKER_01
I m not worried about becoming too cynical.
1
SPEAKER_01
I think I m like the extreme opposite of a cynical person, but I m worried about just becoming like less of a default trusting person.
1
SPEAKER_02
I m actually not sure which mode is best to operate in for a person
1
SPEAKER_02
who s developing AGI, trusting or untrusting.
1
SPEAKER_02
It s an interesting journey you re on.
1
SPEAKER_02
But in terms of structure, see, I m more interested on the human level.
1
SPEAKER_02
Like, how do you surround yourself with humans that are building cool shit, but also are making wise decisions?
1
SPEAKER_02
Because the more money you start making, the more power the thing has, the weirder people get.
1
SPEAKER_01
You know, I think you could like, you could make all kinds of comments about
1
SPEAKER_01
board members and the level of trust I should have had there or how I should have done things differently.
1
SPEAKER_01
But in terms of the team here, I think you d have to give me a very good grade on that one.
1
SPEAKER_01
And I have just enormous gratitude and trust and respect for the people that I work with every day.
1
SPEAKER_00
And I think being surrounded with people like that is really important.
1
SPEAKER_02
Our mutual friend Elon sued OpenAI.
1
SPEAKER_02
What is the essence of what he s criticizing?
1
SPEAKER_02
To what degree does he have a point?
1
SPEAKER_02
To what degree is he wrong?
1
SPEAKER_01
I don t know what it s really about.
1
SPEAKER_01
We started off
1
SPEAKER_01
just thinking we were gonna be a research lab and having no idea about how this technology was gonna go.
1
SPEAKER_01
It s hard to, because it was only seven or eight years ago, it s hard to go back and really remember what it was like then.
1
SPEAKER_01
But this was before language models were a big deal.
1
SPEAKER_01
This was before we had any idea about an API or selling access to a chatbot.
1
SPEAKER_01
This was before we had any idea we were gonna productize at all.
1
SPEAKER_01
So we re like, we re just gonna try to do research and we don t really know what we re gonna do with that.
1
SPEAKER_01
I think with many fundamentally new things, you start fumbling through the dark and you make some assumptions, most of which turn out to be wrong.
1
SPEAKER_00
Then it became clear that we were going to need to do
1
SPEAKER_00
different things and also have huge amounts more capital.
1
SPEAKER_01
We said, okay, well, the structure doesn t quite work for that.
1
SPEAKER_01
How do we patch the structure?
1
SPEAKER_01
Then you patch it again and patch it again, and you end up with something that does look kind of eyebrow-raising to say the least, but we got here gradually with, I think, reasonable decisions at each point along the way.
1
SPEAKER_01
It doesn t mean I wouldn t do it totally differently if we could go back now with an oracle, but you don t get the oracle at the time.
1
SPEAKER_01
But anyway, in terms of what Elon s real motivations here are, I don t know.
1
SPEAKER_02
To the degree you remember, what was the response that OpenAI gave in the blog post?
1
SPEAKER_02
Can you summarize it?
1
SPEAKER_01
Oh, we just said like,
1
SPEAKER_01
you know, Elon said this set of things, here s our characterization, or here s the sort of, not our characterization, here s like the characterization of how this went down.
1
SPEAKER_01
We tried to like not make it emotional and just sort of say like, here s the history.
1
SPEAKER_02
there s a degree of mischaracterization from Elon here about one of the points you just made, which is the degree of uncertainty you had at the time.
1
SPEAKER_02
You guys are a bunch of like, a small group of researchers crazily talking about AGI when everybody s laughing at that thought.
1
SPEAKER_01
Wasn t that long ago Elon was crazily talking about launching rockets?
1
SPEAKER_00
Yeah.
1
SPEAKER_01
When people were laughing at that thought?
1
SPEAKER_02
So I think he d have more empathy for this.
1
SPEAKER_02
I mean, I do think that there s personal stuff here, that there was a split, that OpenAI and a lot of amazing people here chose to part ways with Elon.
1
SPEAKER_02
Elon chose to part ways.
1
SPEAKER_02
Can you describe that exactly?
1
SPEAKER_01
The choosing to part ways?
1
SPEAKER_01
He thought OpenAI was going to fail.
1
SPEAKER_01
He wanted total control to sort of turn it around.
1
SPEAKER_01
We wanted to keep going in the direction that now has become OpenAI.
1
SPEAKER_01
He also wanted Tesla to be able to build an AGI effort.
1
SPEAKER_01
At various times he wanted to make OpenAI into a for-profit company.
2
SPEAKER_01
that he could have control of or have it merge with Tesla.
2
SPEAKER_01
We didn t wanna do that and he decided to leave, which that s fine.
2
SPEAKER_02
So you re saying, and that s one of the things that the blog post says, is that he wanted OpenAI to be basically acquired by Tesla?
2
SPEAKER_02
Yeah.
2
SPEAKER_02
In the same way that, or maybe something similar or maybe something more dramatic than the partnership with Microsoft?
2
SPEAKER_01
My memory is the proposal was just like, yeah, get acquired by Tesla and have Tesla have full control over it.
2
SPEAKER_01
I m pretty sure that s what it was.
2
SPEAKER_02
So what is the word open in OpenAI mean?
2
SPEAKER_02
To Elon at the time, Ilya has talked about this in the email exchanges and all this kind of stuff.
2
SPEAKER_02
What does it mean to you at the time?
2
SPEAKER_02
What does it mean to you now?
2
SPEAKER_01
I would definitely pick a different, speaking of going back with an Oracle, I d pick a different name.
2
SPEAKER_01
One of the things that I think OpenAI is doing that is the most important of everything that we re doing is putting powerful technology in the hands of people for free as a public good.
2
SPEAKER_01
We don t run ads on our free version.
2
SPEAKER_01
We don t monetize it in other ways.
2
SPEAKER_01
We just say it s part of our mission.
2
SPEAKER_01
We want to put increasingly powerful tools in the hands of people for free and get them to use them.
2
SPEAKER_01
I think
2
SPEAKER_01
That kind of open is really important to our mission.
2
SPEAKER_01
I think if you give people great tools and teach them to use them or don t even teach them, they ll figure it out and let them go build an incredible future for each other with that, that s a big deal.
2
SPEAKER_01
If we can keep putting free or low-cost or free and low-cost powerful AI tools out in the world, I think that s a huge deal for how we fulfill the mission.
2
SPEAKER_01
Open source or not, yeah, I think we should open source some stuff and not other stuff.
2
SPEAKER_01
It does become this religious battle line where nuance is hard to have, but I think nuance is the right answer.
2
SPEAKER_02
So he said, change your name to Closed AI and I ll drop the lawsuit.
2
SPEAKER_02
I mean, is it going to become this battleground in the land of memes about the name?
2
SPEAKER_00
I think that speaks to the seriousness with which Elon means the lawsuit.
2
SPEAKER_00
And that s like an astonishing thing to say, I think.
2
SPEAKER_02
I don t think the lawsuit, maybe correct me if I m wrong, but I don t think the lawsuit is legally serious.
2
SPEAKER_02
It s more to make a point about the future of AGI and the company that s currently leading the way.
2
SPEAKER_01
So, look, I mean, Grok had not open sourced anything until people pointed out it was a little bit hypocritical, and then he announced that Grok will open source things this week.
2
SPEAKER_01
I don t think open source versus not is what this is really about for him.
2
SPEAKER_02
Well, we ll talk about open source and not.
2
SPEAKER_02
I do think maybe criticizing the competition is great.
2
SPEAKER_02
Just talking a little shit, that s great.
2
SPEAKER_02
But friendly competition versus like,
2
SPEAKER_02
I personally hate lawsuits.
2
SPEAKER_01
Look, I think this whole thing is like unbecoming of a builder, and I respect Elon as one of the great builders of our time.
2
SPEAKER_01
I know he knows what it s like to have like haters attack him and it makes me extra sad he s doing it to us.
2
SPEAKER_02
Yeah, he s one of the greatest builders of all time.
2
SPEAKER_02
Potentially the greatest builder of all time.
2
SPEAKER_01
It makes me sad.
2
SPEAKER_01
I think it makes a lot of people sad.
2
SPEAKER_01
Like there s a lot of people who ve really looked up to him for a long time and said this.
2
SPEAKER_01
I said, you know, in some interview or something that I missed the old Elon and the number of messages I got being like that exactly encapsulates how I feel.
2
SPEAKER_02
I think he should just win.
2
SPEAKER_02
He should just make Grok beat GPT, and then GPT beats Grok, and it s just a competition.
2
SPEAKER_02
And it s beautiful for everybody.
2
SPEAKER_02
But on the question of open source, do you think there s a lot of companies playing with this idea?
2
SPEAKER_02
It s quite interesting.
2
SPEAKER_02
I would say
2
SPEAKER_02
meta, surprisingly, has led the way on this, or at least took the first step in the game of chess of really open sourcing the model.
2
SPEAKER_02
Of course, it s not the state-of-the-art model, but open sourcing llama.
2
SPEAKER_02
Google is flirting with the idea of open sourcing a smaller version.
2
SPEAKER_02
What are the pros and cons of open sourcing?
2
SPEAKER_01
played around with this idea?
2
SPEAKER_01
Yeah, I think there is definitely a place for open source models, particularly smaller models that people can run locally.
2
SPEAKER_01
I think there s huge demand for.
2
SPEAKER_01
I think there will be some open source models, there will be some closed source models.
2
SPEAKER_01
It won t be unlike other ecosystems in that way.
2
SPEAKER_02
I listened to an all-in podcast talking about this lawsuit and all that kind of stuff, and they were more concerned about the precedent of going from non-profit to this cap for profit.
2
SPEAKER_02
what precedent to set for other startups?
2
SPEAKER_01
Is that something?
2
SPEAKER_01
I don t, I would heavily discourage any startup that was thinking about starting as a non-profit and adding like a for-profit arm later.
2
SPEAKER_01
I d heavily discourage them from doing that.
2
SPEAKER_01
I don t think we ll set a precedent here.
2
SPEAKER_01
Okay, so most startups should go just- For sure.
2
SPEAKER_01
And again, if we knew what was gonna happen, we would have done that too.
2
SPEAKER_02
Well, in theory, if you dance beautifully here, there s some tax incentives or whatever.
2
SPEAKER_02
I don t think that s how most people think about these things.
2
SPEAKER_02
It s just not possible to save a lot of money for a startup if you do it this way.
2
SPEAKER_02
No, I think there s like laws that would make that pretty difficult.
2
SPEAKER_02
What do you hope this goes with Elon?
2
SPEAKER_02
This tension, this dance, what do you hope this, like if we go one, two, three years from now, your relationship with him on a personal level too, like friendship, friendly competition, just all this kind of stuff.
2
SPEAKER_00
Yeah, I mean, I really respect Elon.
2
SPEAKER_00
And I hope that years in the future we have an amicable relationship.
2
SPEAKER_02
Yeah, I hope you guys have an amicable relationship like this month.
2
SPEAKER_02
And just compete and win and explore these ideas together.
2
SPEAKER_02
I do suppose there s competition for talent or whatever, but it should be friendly competition.
2
SPEAKER_02
Just build cool shit.
2
SPEAKER_02
And Elon is pretty good at building cool shit, but so are you.
2
SPEAKER_02
So speaking of cool shit, Sora, there s like a million questions I could ask.
2
SPEAKER_02
First of all, it s amazing.
2
SPEAKER_02
It truly is amazing on a product level, but also just on a philosophical level.
2
SPEAKER_02
So let me just technical slash philosophical ask, what do you think it understands about the world?
2
SPEAKER_02
more or less than GPT-4, for example.
2
SPEAKER_02
The world model, when you train on these patches versus language tokens.
2
SPEAKER_01
I think all of these models understand.
2
SPEAKER_01
something more about the world model than most of us give them credit for.
2
SPEAKER_01
Because there are also very clear things they just don t understand or don t get right, it s easy to look at the weaknesses, see through the veil, and say, ah, this is all fake.
2
SPEAKER_01
But it s not all fake.
2
SPEAKER_01
It s just some of it works and some of it doesn t work.
2
SPEAKER_01
I remember when I started first watching Sora videos, and I would see a person walk in front of something for a few seconds and occlude it, and then walk away, and the same thing was still there.
2
SPEAKER_01
I was like, oh, this is pretty good.
2
SPEAKER_01
There s examples where the underlying physics looks so well-represented over a lot of steps in a sequence.
2
SPEAKER_01
It s like, oh, this is quite impressive.
2
SPEAKER_01
But fundamentally, these models are just getting better, and that will keep happening.
2
SPEAKER_01
If you look at the trajectory from Dali 1 to 2 to 3 to Sora, there are a lot of people that were dunked on each version, saying, it can t do this, it can t do that, and look at it now.
2
SPEAKER_02
The thing you just mentioned with occlusions is basically modeling the physics, the three-dimensional physics of the world sufficiently well to capture those kinds of things.
2
SPEAKER_02
Well,
2
SPEAKER_02
Or like, yeah, maybe you can tell me, in order to deal with occlusions, what does the world model need to?
2
SPEAKER_01
Yeah, so what I would say is it s doing something to deal with occlusions really well.
2
SPEAKER_01
What I represent that it has like a great underlying 3D model of the world, it s a little bit more of a stretch.
2
SPEAKER_02
But can you get there through just these kinds of two-dimensional training data approaches?
2
SPEAKER_01
It looks like this approach is going to go surprisingly far.
2
SPEAKER_01
I don t want to speculate too much about what limits it will surmount and which it won t, but... What are some interesting limitations of the system that you ve seen?
2
SPEAKER_02
I mean, there s been some fun ones you ve posted.
2
SPEAKER_01
There s all kinds of fun.
2
SPEAKER_01
I mean, like, you know, cats sprouting an extra limit, random points in a video.
2
SPEAKER_01
Pick what you want, but there s still a lot of problems, a lot of weaknesses.
2
SPEAKER_02
Do you think there s a fundamental flaw of the approach?
2
SPEAKER_02
Or is it just, you know, bigger model or better?
2
SPEAKER_02
like technical details or better data, more data is going to solve the cat sprouting?
2
SPEAKER_02
I would say yes to both.
2
SPEAKER_01
Like I think there is something about the approach which just seems to feel different from how we think and learn and whatever.
2
SPEAKER_01
And then also I think it ll get better with scale.
2
SPEAKER_02
Like I mentioned, LLMs have tokens, text tokens, and Sora has visual patches.
2
SPEAKER_02
So it converts all visual data, diverse kinds of visual data, videos and images into patches.
2
SPEAKER_02
Is the training to the degree you can say fully self-supervised?
2
SPEAKER_02
Or is there some manual labeling going on?
2
SPEAKER_02
Like, what s the involvement of humans in all this?
2
SPEAKER_01
I mean, without saying anything specific about the SOAR approach, we use lots of human data in our work.
2
SPEAKER_02
But not internet-scale data.
2
SPEAKER_02
So, lots of humans.
2
SPEAKER_02
Lots is a complicated word, Sam.
2
SPEAKER_02
Well, I think lots is a fair word in this case.
2
SPEAKER_02
But it doesn t, because to me, lots, like listen, I m an introvert, and when I hang out with like three people, that s a lot of people.
2
SPEAKER_02
Four people, that s a lot.
2
SPEAKER_02
But I suppose you mean more than?
2
SPEAKER_02
More than three people work on labeling the data for these models, yeah.
2
SPEAKER_02
Okay, all right.
2
SPEAKER_02
But fundamentally, there s a lot of self-supervised learning.
2
SPEAKER_02
Because what you mentioned in the technical report is internet-scale data.
2
SPEAKER_02
That s another beautiful, it s like poetry.
2
SPEAKER_02
So it s a lot of data that s not human label, it s like, it s self-supervised in that way.
2
SPEAKER_02
And then the question is how much data is there on the internet that could be used in this, that is conducive to this kind of self-supervised way, if only we knew the details of the self-supervised.
2
SPEAKER_02
Have you considered opening it up a little more in details?
2
SPEAKER_02
We have, you mean for SOAR specifically?
2
SPEAKER_02
SOAR specifically, because it s so interesting that, like, can the same magic of LLMs now start moving towards visual data?
2
SPEAKER_02
And what does that take to do that?
2
SPEAKER_01
I mean, it looks to me like yes, but we have more work to do.
2
SPEAKER_02
Sure.
2
SPEAKER_02
What are the dangers?
2
SPEAKER_02
Why are you concerned about releasing the system?
2
SPEAKER_02
What are some possible dangers of this?
2
SPEAKER_01
I mean, frankly speaking, one thing we have to do before releasing the system is just like get it to work at
2
SPEAKER_01
a level of efficiency that will deliver the scale people are going to want from this.
2
SPEAKER_01
So I don t want to like downplay that.
2
SPEAKER_01
And there s still a ton, ton of work to do there.
2
SPEAKER_01
But you know, you can imagine like
2
SPEAKER_01
issues with deep fakes, misinformation.
2
SPEAKER_01
We try to be a thoughtful company about what we put out into the world, and it doesn t take much thought to think about the ways this can go badly.
2
SPEAKER_02
There s a lot of tough questions here.
2
SPEAKER_02
You re dealing in a very tough space.
2
SPEAKER_02
Do you think training AI should be or is fair use under copyright law?
2
SPEAKER_01
I think the question behind that question is, do people who create valuable data deserve to have some way that they get compensated for use of it?
2
SPEAKER_01
And that, I think the answer is yes.
2
SPEAKER_01
I don t know yet what the answer is.
2
SPEAKER_01
People have proposed a lot of different things.
2
SPEAKER_01
We ve tried some different models.
2
SPEAKER_01
But if I m an artist, for example,
2
SPEAKER_01
A, I would like to be able to opt out of people generating art in my style, and B, if they do generate art in my style, I d like to have some economic model associated with that.
2
SPEAKER_02
Yeah, it s that transition from CDs to Napster to Spotify.
2
SPEAKER_02
You have to figure out some kind of model.
2
SPEAKER_01
The model changes, but people have got to get paid.
2
SPEAKER_02
There should be some kind of incentive, if we zoom out even more, for humans to keep doing cool shit.
2
SPEAKER_01
Everything I worry about, humans are going to do cool shit and society is going to find some way to reward it.
2
SPEAKER_01
That seems pretty hardwired.
2
SPEAKER_01
We want to create, we want to be useful, we want to achieve status in whatever way.
2
SPEAKER_01
That s not going anywhere, I don t think.
2
SPEAKER_02
But the reward might not be monetary, financial.
2
SPEAKER_02
It might be like fame and celebration of other cool people.
2
SPEAKER_01
Maybe financial in some other way.
2
SPEAKER_01
Again, I don t think we ve seen the last evolution of how the economic system s going to work.
2
SPEAKER_02
Yeah, but artists and creators are worried.
2
SPEAKER_02
When they see Sora, they re like, holy shit.
2
SPEAKER_02
Sure.
2
SPEAKER_01
Artists were also super worried when photography came out.
2
SPEAKER_02
Yeah.
2
SPEAKER_01
And then photography became a new art form and people made a lot of money taking pictures.
2
SPEAKER_01
And I think things like that will keep happening.
2
SPEAKER_01
People will use the new tools in new ways.
2
SPEAKER_02
If we just look on YouTube or something like this, how much of that will be using Sora like
2
SPEAKER_02
AI-generated content, do you think, in the next five years?
2
SPEAKER_01
People talk about how many jobs they re going to do in five years.
2
SPEAKER_01
The framework that people have is what percentage of current jobs are just going to be totally replaced by some AI doing the job.
2
SPEAKER_01
The way I think about it is not what percent of jobs AI will do, but what percent of tasks will AI do and over what time horizon.
2
SPEAKER_01
If you think of all of the five-second tasks in the economy, the five-minute tasks, the five-hour tasks, maybe even the five-day tasks, how many of those can AI do?
2
SPEAKER_01
I think that s a way more interesting, impactful, important question than how many jobs AI can do because it is a tool that will work at increasing levels of sophistication and over longer and longer time horizons for more and more tasks and let people operate at a higher level of abstraction.
2
SPEAKER_01
Maybe people are way more efficient at the job they do.
2
SPEAKER_01
At some point,
2
SPEAKER_01
That s not just a quantitative change, but it s a qualitative one, too, about the kinds of problems you can keep in your head.
2
SPEAKER_01
I think that for videos on YouTube, it ll be the same.
2
SPEAKER_01
Many videos, maybe most of them, will use AI tools in the production, but they ll still be fundamentally driven by a person thinking about it, putting it together, doing parts of it, sort of directing and running it.
2
SPEAKER_02
Yeah, it s so interesting.
2
SPEAKER_02
I mean, it s scary, but it s interesting to think about.
2
SPEAKER_02
I tend to believe that humans like to watch other humans or other human... Humans really care about other humans a lot.
2
SPEAKER_02
Yeah.
2
SPEAKER_02
If there s a cooler thing that s more, that s better than a human, humans care about that for like two days and then they go back to humans.
2
SPEAKER_02
That seems very deeply wired.
2
SPEAKER_02
It s the whole chess thing.
2
SPEAKER_02
Yeah, but now let s everybody keep playing.
2
SPEAKER_02
And let s ignore the elephant in the room that humans are really bad at chess relative to AI systems.
2
SPEAKER_02
We still run races and cars are much faster.
2
SPEAKER_01
I mean, there s like a lot of examples.
2
SPEAKER_02
Yeah, and maybe it ll just be tooling like in the Adobe suite type of way where you can just make videos much easier and all that kind of stuff.
2
SPEAKER_02
Listen, I hate being in front of the camera.
2
SPEAKER_02
If I can figure out a way to not be in front of the camera, I would love it.
2
SPEAKER_02
Unfortunately, it ll take a while.
2
SPEAKER_02
Like that, generating faces, it s getting there, but generating faces in video format is tricky when it s specific people versus generic people.
2
SPEAKER_02
Let me ask you about GPT-4.
2
SPEAKER_02
There s so many questions.
2
SPEAKER_02
First of all, also amazing.
2
SPEAKER_02
Looking back, it ll probably be this kind of historic pivotal moment with 3, 5, and 4 with Chad GPT.
2
SPEAKER_02
Maybe 5 will be the pivotal moment, I don t know.
2
SPEAKER_02
Hard to say that looking forwards.
2
SPEAKER_02
We never know.
2
SPEAKER_02
That s the annoying thing about the future.
2
SPEAKER_02
It s hard to predict.
2
SPEAKER_02
But for me, looking back, GPT-4, Chad GPT is pretty damn impressive.
2
SPEAKER_02
Like, historically impressive.
2
SPEAKER_02
So, allow me to ask, what s been the most impressive capabilities of GPT-4 to you and GPT-4 Turbo?
2
SPEAKER_02
I think it kind of sucks.
2
SPEAKER_02
Typical human also, gotten used to an awesome thing.
2
SPEAKER_01
No, I think it is an amazing thing, but relative to where we need to get to and where I believe we will
3
SPEAKER_00
got to.
3
SPEAKER_00
At the time of GPT-3, people were like, oh, this is amazing.
3
SPEAKER_00
This is this marvel of technology, and it is.
3
SPEAKER_00
It was.
3
SPEAKER_00
But now we have GPT-4 and look at GPT-3 and you re like, that s unimaginably horrible.
3
SPEAKER_00
I expect that the delta between five and four will be the same as between four and three.
3
SPEAKER_00
I think it is our job to
3
SPEAKER_00
live a few years in the future and remember that the tools we have now are gonna kind of suck looking backwards at them.
3
SPEAKER_00
And that s how we make sure the future is better.
3
SPEAKER_02
What are the most glorious ways that GPT-4 sucks?
3
SPEAKER_00
Meaning... What are the best things it can do?
3
SPEAKER_02
What are the best things it can do and the limits of those best things that allow you to say it sucks, therefore gives you inspiration and hope for the future.
3
SPEAKER_00
You know, one thing I ve been using it for more recently is sort of like a brainstorming partner.
3
SPEAKER_00
And there s a glimmer of something amazing in there.
3
SPEAKER_00
I don t think it gets, you know, when people talk about it, what it does, they re like, oh, it helps me code more productively.
3
SPEAKER_00
It helps me write more faster and better.
3
SPEAKER_00
It helps me translate from this language to another, all these like amazing things.
3
SPEAKER_00
There s something about the creative brainstorming partner – I need to come up with a name for this thing.
3
SPEAKER_00
I need to think about this problem in a different way.
3
SPEAKER_00
I m not sure what to do here – that I think gives a glimpse of something I hope to see more of.
3
SPEAKER_00
One of the other things that you can see a very small glimpse of is
3
SPEAKER_00
When I can help on longer horizon tasks, you know, break down something into multiple steps, maybe like execute some of those steps, search the internet, write code, whatever, put that together.
3
SPEAKER_00
When that works, which is not very often, it s like very magical.
3
SPEAKER_02
The iterative back and forth with a human.
3
SPEAKER_00
It works a lot for me.
3
SPEAKER_00
What do you mean?
3
SPEAKER_00
Iterative back and forth with a human, when it can go do like a 10-step problem on its own.
3
SPEAKER_00
Oh.
3
SPEAKER_00
Doesn t work for that too often, sometimes.
3
SPEAKER_00
At multiple layers of abstraction, or do you mean just sequential?
3
SPEAKER_00
Both, like, you know, to break it down and then do things at different layers of abstraction and put them together.
3
SPEAKER_00
Look, I don t wanna like downplay the accomplishment of GPT-4.
3
SPEAKER_00
But I don t want to overstate it either.
3
SPEAKER_00
And I think this point that we are on an exponential curve, we will look back relatively soon at GPT-4 like we look back at GPT-3 now.
3
SPEAKER_02
That said, I mean, Chat-GBT was a transition to where people started to believe, there was a kind of, there is an uptick of believing.
3
SPEAKER_02
Not internally at OpenAI, perhaps, there s believers here, but when you think about Google.
3
SPEAKER_00
And in that sense, I do think it ll be a moment where a lot of the world went from not believing to believing.
3
SPEAKER_00
That was more about the Chat-GBT interface
3
SPEAKER_00
And by the interface and product, I also mean the post-training of the model and how we tune it to be helpful to you and how to use it, then the underlying model itself.
3
SPEAKER_02
How much of those two, each of those things are important, the underlying model and the RLHF or something of that nature that tunes it to be more compelling to the human, more effective and productive for the human?
3
SPEAKER_00
I mean, they re both super important, but the RLHF, the post-training step, the little wrapper of things that, from a compute perspective, little wrapper of things that we do on top of the base model, even though it s a huge amount of work, that s really important to say nothing of the product that we build around it.
3
SPEAKER_00
In some sense, we did have to do two things.
3
SPEAKER_00
We had to invent the underlying technology, and then we had to figure out
3
SPEAKER_00
how to make it into a product people would love, which is not just about the actual product work itself, but this whole other step of how you align and make it useful.
3
SPEAKER_02
Trevor Burrus And how you make the scale work where a lot of people can use it at the same time, all that kind of stuff.
3
SPEAKER_00
That was an unknown difficult thing.
3
SPEAKER_00
We knew we were going to have to scale it up.
3
SPEAKER_00
We had to go do two things that had never been done before that were both, I would say, quite significant achievements and then a lot of things like scaling it up that other companies have had to do before.
3
SPEAKER_02
How does the context window of going from 8K to 128K tokens compare from GPT-4 to GPT-4 Turbo?
3
SPEAKER_00
Most people don t need all the way to 128K most of the time.
3
SPEAKER_00
Although, you know, if we dream into the distant future, we ll have like way distant future, we ll have like context length of several billion.
3
SPEAKER_00
You will feed in all of your information, all of your history over time, and it ll just get to know you better and better.
3
SPEAKER_00
And that ll be great.
3
SPEAKER_00
For now, the way people use these models, they re not doing that.
3
SPEAKER_00
People sometimes post in a paper or a significant fraction of a code repository or whatever, but most usage of the models is not using the long context most of the time.
3
SPEAKER_02
I like that this is your I have a dream speech.
3
SPEAKER_02
One day you ll be judged by the full context of your character, or of your whole lifetime.
3
SPEAKER_02
That s interesting.
3
SPEAKER_02
So that s part of the expansion that you re hoping for, is a greater and greater context.
3
SPEAKER_00
I saw this internet clip once.
3
SPEAKER_00
I m going to get the numbers wrong, but it was like Bill Gates talking about the amount of memory on some early computer.
3
SPEAKER_00
Maybe 64K, maybe 640K, something like that.
3
SPEAKER_00
Most of it was used for the screen buffer.
3
SPEAKER_00
He just couldn t seem genuine as couldn t imagine that the world would eventually need gigabytes of memory in a computer or terabytes of memory in a computer.
3
SPEAKER_00
And you always do.
3
SPEAKER_00
Or you always do just need to like follow the exponential of technology.
3
SPEAKER_00
And we re gonna like, we will find out how to use better technology.
3
SPEAKER_00
So I can t really imagine what it s like right now for context links to go out to the billions someday.
3
SPEAKER_00
And they might not literally go there, but effectively it ll feel like that.
3
SPEAKER_00
But I know we ll use it and really not wanna go back once we have it.
3
SPEAKER_02
Yeah, even saying billions 10 years from now might seem dumb because it ll be like,
3
SPEAKER_02
trillions upon trillions.
3
SPEAKER_02
There d be some kind of breakthrough that will effectively feel like infinite context.
3
SPEAKER_02
But even 120, I have to be honest, I haven t pushed it to that degree.
3
SPEAKER_02
Maybe putting in entire books or like parts of books and so on, papers.
3
SPEAKER_02
What are some interesting use cases of GPT-4 that you ve seen?
3
SPEAKER_00
The thing that I find most interesting is not any particular use case that we can talk about those, but it s people who kind of like
3
SPEAKER_00
This is mostly younger people, but people who use it as their default start for any kind of knowledge work task.
3
SPEAKER_00
And it s the fact that it can do a lot of things reasonably well.
3
SPEAKER_00
You can use GPTV, you can use it to help you write code, you can use it to help you do search, you can use it to edit a paper.
3
SPEAKER_00
The most interesting thing to me is the people who just use it as the start of their workflow.
3
SPEAKER_02
I do as well for many things, like I use it as a reading partner for reading books.
3
SPEAKER_02
It helps me think, help me think through ideas, especially when the books are classics, so it s really well written about and it actually is as
3
SPEAKER_02
I find it often to be significantly better than even Wikipedia on well-covered topics.
3
SPEAKER_02
It s somehow more balanced and more nuanced.
3
SPEAKER_02
Or maybe it s me, but it inspires me to think deeper than a Wikipedia article does.
3
SPEAKER_02
I m not exactly sure what that is.
3
SPEAKER_02
You mentioned this collaboration.
3
SPEAKER_02
I m not sure where the magic is, if it s in here or if it s in there, or if it s somewhere in between, I m not sure.
3
SPEAKER_02
But one of the things that concerns me for knowledge tasks when I start with GPT is,
3
SPEAKER_02
I ll usually have to do fact-checking after.
3
SPEAKER_02
Like, check that it didn t come up with fake stuff.
3
SPEAKER_02
How do you figure that out?
3
SPEAKER_02
That, you know, GPT can come up with fake stuff that sounds really convincing.
3
SPEAKER_02
So how do you ground it in truth?
3
SPEAKER_00
That s obviously an area of intense interest for us.
3
SPEAKER_00
I think it s going to get a lot better with upcoming versions, but we ll have to continue to work on it, and we re not going to have it all solved this year.
3
SPEAKER_02
Well, the scary thing is, as it gets better, you ll start not doing the fact-checking more and more, right?
3
SPEAKER_00
I m of two minds about that.
3
SPEAKER_00
I think people are like much more sophisticated users of technology than we often give them credit for.
3
SPEAKER_00
And people seem to really understand that GPT, any of these models, hallucinate some of the time.
3
SPEAKER_00
And if it s mission-critical, you got to check it.
3
SPEAKER_02
Except journalists don t seem to understand that.
3
SPEAKER_00
I ve seen journalists half-assedly just using GPT for... Of the long list of things I d like to dunk on journalists for, this is not my top criticism of them.
3
SPEAKER_02
Well, I think the bigger criticism is perhaps the pressures and the incentives of being a journalist is that you have to work really quickly and this is a shortcut.
3
SPEAKER_02
I would love our society to incentivize like
3
SPEAKER_02
I would too.
3
SPEAKER_02
A long, like a journalist, journalistic efforts that take days and weeks and rewards great in-depth journalism.
3
SPEAKER_02
Also journalism that presents stuff in a balanced way where it s like celebrates people while criticizing them even though the criticism is the thing that gets clicks.
3
SPEAKER_02
And making shit up also gets clicks.
3
SPEAKER_02
And headlines that mischaracterize completely.
3
SPEAKER_02
I m sure you have a lot of people dunking on, well all that drama probably got a lot of clicks.
3
SPEAKER_02
Probably did.
3
SPEAKER_02
And that s a bigger problem about human civilization.
3
SPEAKER_02
I d love to see solved.
3
SPEAKER_02
This is where we celebrate a bit more.
3
SPEAKER_02
You ve given Chad GPT the ability to have memories.
3
SPEAKER_02
You ve been playing with that about previous conversations.
3
SPEAKER_02
and also the ability to turn off memory.
3
SPEAKER_02
I wish I could do that sometimes, just turn on and off, depending, I guess sometimes alcohol can do that, but not optimally, I suppose.
3
SPEAKER_02
What have you seen through that, like playing around with that idea of remembering conversations or not?
3
SPEAKER_00
We re very early in our explorations here, but I think what people want, or at least what I want for myself, is a model that gets to know me and gets more useful to me over time.
3
SPEAKER_00
This is an early exploration.
3
SPEAKER_00
I think there s like a lot of other things to do, but that s where we d like to head.
3
SPEAKER_00
You d like to use a model and over the course of your life, or use a system, there ll be many models, and over the course of your life, it gets better and better.
3
SPEAKER_02
Yeah, how hard is that problem?
3
SPEAKER_02
Because right now it s more like remembering little factoids and preferences and so on.
3
SPEAKER_02
What about remembering, like, don t you want GPT to remember all the shit you went through in November?
3
SPEAKER_02
and all the drama, and then you can, because right now you re clearly blocking it out a little bit.
3
SPEAKER_00
It s not just that I want it to remember that.
3
SPEAKER_00
I want it to integrate the lessons of that and remind me in the future what to do differently or what to watch out for.
3
SPEAKER_00
And we all gain from experience over the course of our lives, varying degrees.
3
SPEAKER_00
And I d like my AI agent to gain with that experience too.
3
SPEAKER_00
So if we go back and let ourselves imagine that trillions and trillions of contacts length,
3
SPEAKER_00
If I can put every conversation I ve ever had with anybody in my life in there, if I can have all of my emails input out, like all of my input output in the context window every time I ask a question, that d be pretty cool, I think.
3
SPEAKER_02
Yeah, I think that would be very cool.
3
SPEAKER_02
People sometimes will hear that and be concerned about privacy.
3
SPEAKER_02
What do you think about that?
3
SPEAKER_02
that aspect of it, the more effective the AI becomes at really integrating all the experiences and all the data that happened to you and giving you advice.
3
SPEAKER_00
I think the right answer there is just user choice.
3
SPEAKER_00
You know, anything I want stricken from the record from my AI agent, I want to be able to take out.
3
SPEAKER_00
If I don t want it to remember anything, I want that too.
3
SPEAKER_00
you and I may have different opinions about where on that privacy utility trade-off for our own AI we want to be, which is totally fine.
3
SPEAKER_00
But I think the answer is just like really easy user choice.
3
SPEAKER_02
But there should be some
3
SPEAKER_02
high level of transparency from a company about the user choice.
3
SPEAKER_02
Because sometimes companies in the past have been kind of shady about like, yeah, it s kind of presumed that we re collecting all your data and we re using it for a good reason, for advertisement and so on.
3
SPEAKER_02
But there s not a transparency about the details of that.
3
SPEAKER_00
That s totally true.
3
SPEAKER_00
You know, you mentioned earlier that I m like blocking out the November stuff.
3
SPEAKER_01
I m just teasing you.
3
SPEAKER_00
Well, I mean, I think it was
3
SPEAKER_00
a very traumatic thing, and it did immobilize me for a long period of time.
3
SPEAKER_00
Definitely the hardest work that I ve had to do was just keep working that period, because I had to try to come back in here and put the pieces together while I was just in shock and pain.
3
SPEAKER_00
Nobody really cares about that.
3
SPEAKER_00
The team gave me a pass, and I was not working at my normal level.
3
SPEAKER_00
But there was a period where I was just like,
3
SPEAKER_00
It was really hard to have to do both, but I kind of woke up one morning and I was like, this was a horrible thing that happened to me.
3
SPEAKER_00
I think I could just feel like a victim forever.
3
SPEAKER_00
Or I can say this is like the most important work I ll ever touch in my life and I need to get back to it.
3
SPEAKER_00
And it doesn t mean that I ve repressed it because sometimes I like wake up in the middle of the night thinking about it, but I do feel like an obligation to keep moving forward.
3
SPEAKER_02
That s beautifully said, but there could be some lingering stuff in there, like what I would be concerned about is...
3
SPEAKER_02
That trust thing that you mentioned, that being paranoid about people as opposed to just trusting everybody or most people, like using your gut, it s a tricky dance.
3
SPEAKER_02
I mean, because I ve seen in my part-time explorations, I ve been diving deeply into the Zelensky administration and the Putin administration and the dynamics there in wartime in a very highly stressful environment.
3
SPEAKER_02
And what happens is distrust.
3
SPEAKER_02
And you isolate yourself both.
3
SPEAKER_02
And you start to not see the world clearly.
3
SPEAKER_02
And that s a concern, that s a human concern.
3
SPEAKER_02
You seem to have taken it in stride and kind of learned the good lessons and felt the love and let the love energize you.
3
SPEAKER_02
Which is great, but still can linger in there.
3
SPEAKER_02
There s just some questions I would love to ask your intuition about what s GPT able to do and not.
3
SPEAKER_02
So it s allocating approximately the same amount of compute for each token it generates.
3
SPEAKER_02
Is there room there in this kind of approach to slower thinking, sequential thinking?
3
SPEAKER_02
I think there will be a new paradigm for that kind of thinking.
3
SPEAKER_02
Will it be similar like architecturally as what we re seeing now with LLMs?
3
SPEAKER_02
Is it a layer on top of the LLMs?
3
SPEAKER_00
I can imagine many ways to implement that.
3
SPEAKER_00
I think that s less important than the question you were getting at, which is, do we need a way to do a slower kind of thinking where the answer doesn t have to get like, you know, it s like,
3
SPEAKER_00
I guess like spiritually, you could say that you want an AI to be able to think harder about a harder problem and answer more quickly about an easier problem.
3
SPEAKER_00
And I think that will be important.
3
SPEAKER_02
Is that like a human thought that we re just having, you should be able to think hard?
3
SPEAKER_02
Is that wrong intuition?
3
SPEAKER_02
I suspect that s a reasonable intuition.
3
SPEAKER_02
Interesting.
3
SPEAKER_02
So it s not possible once the GPT gets like GPT-7, we ll just be instantaneously be able to see, you know, here s the proof from our theorem.
3
SPEAKER_00
It seems to me like you want to be able to allocate more compute to harder problems.
3
SPEAKER_01
It seems to me that a system knowing – if you ask a system like that, prove Fermat s last theorem versus what s today s date, unless it already knew and had memorized the answer to the
3
SPEAKER_01
proof, assuming it s got to go figure that out.
3
SPEAKER_01
Seems like that will take more compute.
3
SPEAKER_02
But can it look like a basically LLM talking to itself, that kind of thing?
3
SPEAKER_00
Maybe.
3
SPEAKER_00
I mean, there s a lot of things that you could imagine working.
3
SPEAKER_00
What, like, what the right or the best way to do that will be, we don t know.
3
SPEAKER_02
This does make me think of the mysterious, the lore behind
4
SPEAKER_00
Q-Star.
4
SPEAKER_00
What s this mysterious Q-Star project?
4
SPEAKER_00
Is it also in the same nuclear facility?
4
SPEAKER_01
There is no nuclear facility.
4
SPEAKER_00
That s what a person with a nuclear facility always says.
4
SPEAKER_00
I would love to have a secret nuclear facility.
4
SPEAKER_00
There isn t one.
4
SPEAKER_00
Alright.
4
SPEAKER_00
Maybe someday.
4
SPEAKER_00
Someday?
4
SPEAKER_00
Alright.
4
SPEAKER_01
One can dream.
4
SPEAKER_01
OpenAI is not a good company at keeping secrets.
4
SPEAKER_01
It would be nice, you know, we re like been plagued by a lot of leaks and it would be nice if we were able to have something like that.
4
SPEAKER_00
Can you speak to what QSTAR is?
4
SPEAKER_00
We are not ready to talk about that.
4
SPEAKER_00
See, but an answer like that means there s something to talk about.
4
SPEAKER_00
It s very mysterious, Sam.
4
SPEAKER_01
I mean, we work on all kinds of research.
4
SPEAKER_02
We have said for a while that we think
4
SPEAKER_02
better reasoning in these systems is an important direction that we d like to pursue.
4
SPEAKER_02
We haven t cracked the code yet.
4
SPEAKER_00
We re very interested in it.
4
SPEAKER_00
Is there gonna be moments, Q star or otherwise, where there s going to be leaps similar to JadGBT, where you re like?
4
SPEAKER_01
That s a good question.
4
SPEAKER_02
What do I think about that?
4
SPEAKER_01
It s interesting, to me it all feels pretty continuous.
4
SPEAKER_00
Right, this is kind of a theme that you re saying, is there s a gradual, you re basically gradually going up an exponential slope, but from an outsider s perspective, from me just watching it, it does feel like there s leaps.
4
SPEAKER_01
But to you, there isn t.
4
SPEAKER_01
I do wonder if we should have, so, you know, part of the reason that we deploy the way we do is that we think, we call it iterative deployment.
4
SPEAKER_01
Rather than go build in secret until we got all the way to GPT-5, we decided to talk about GPT-1, 2, 3, and 4.
4
SPEAKER_01
Part of the reason there is I think AI and surprise don t go together.
4
SPEAKER_01
Also, the world, people, institutions, whatever you want to call it, need time to adapt and think about these things.
4
SPEAKER_01
I think one of the best things that OpenAI has done is this strategy.
4
SPEAKER_01
We get the world to pay attention to the progress, to take AGI seriously, to think about what
4
SPEAKER_01
systems and structures and governance we want in place before we re under the gun and have to make a rush decision.
4
SPEAKER_01
I think that s really good.
4
SPEAKER_01
But the fact that people like you and others say you still feel like there are these leaps makes me think that maybe we should be doing our releasing even more iteratively.
4
SPEAKER_01
I don t know what that would mean.
4
SPEAKER_01
I don t have an answer ready to go.
4
SPEAKER_01
But our goal is not to have shock updates to the world, the opposite.
4
SPEAKER_00
Yeah, for sure.
4
SPEAKER_00
More iterative would be amazing.
4
SPEAKER_00
I think that s just beautiful for everybody.
4
SPEAKER_01
But that s what we re trying to do.
4
SPEAKER_01
That s like our state of the strategy.
4
SPEAKER_01
And I think we re somehow missing the mark.
4
SPEAKER_01
So maybe we should think about, you know, releasing GPT-5 in a different way or something like that.
4
SPEAKER_00
Yeah, 4.71, 4.72.
4
SPEAKER_00
But people tend to like to celebrate.
4
SPEAKER_00
People celebrate birthdays.
4
SPEAKER_00
I don t know if you know humans, but they kind of have these milestones.
4
SPEAKER_01
I do know some humans.
4
SPEAKER_01
People do like milestones.
4
SPEAKER_01
I totally get that.
4
SPEAKER_01
I think we like milestones too.
4
SPEAKER_01
It s like fun to say, declare victory on this one and go start the next thing.
4
SPEAKER_01
But yeah, I feel like we re somehow getting this a little bit wrong.
4
SPEAKER_00
So when is GPT-5 coming out again?
4
SPEAKER_00
I don t know.
4
SPEAKER_00
That s an honest answer.
4
SPEAKER_00
Oh, that s the honest answer.
4
SPEAKER_00
Is it blink twice if it s this year?
4
SPEAKER_01
I also, we will release,
4
SPEAKER_01
an amazing new model this year.
4
SPEAKER_00
I don t know what we ll call it.
4
SPEAKER_00
So that goes to the question of like, what s the way we release this thing?
4
SPEAKER_00
We ll release over in the coming months.
4
SPEAKER_01
many different things.
4
SPEAKER_01
I think they ll be very cool.
4
SPEAKER_01
I think before we talk about a GPT-5-like model called that or not called that or a little bit worse or a little bit better than what you d expect from a GPT-5, I know we have a lot of other important things to release first.
4
SPEAKER_00
I don t know what to expect from GPT-5.
4
SPEAKER_00
You re making me nervous and excited.
4
SPEAKER_00
What are some of the biggest challenges and bottlenecks to overcome for whatever it ends up being called, but let s call it GPT-5.
4
SPEAKER_00
Just interesting to ask.
4
SPEAKER_00
Is it on the compute side?
4
SPEAKER_00
Is it on the technical side?
4
SPEAKER_01
You know, what s the one big unlock?
4
SPEAKER_01
Is it a bigger computer?
4
SPEAKER_01
Is it like a new secret?
4
SPEAKER_01
Is it something else?
4
SPEAKER_01
It s all of these things together.
4
SPEAKER_01
Like the thing that OpenAI, I think, does really well.
4
SPEAKER_01
This is actually an original Ilio quote that I m going to butcher, but it s something like we multiply 200 medium-sized things together into one giant thing.
4
SPEAKER_00
So there s this distributed constant innovation happening.
4
SPEAKER_00
Yeah.
4
SPEAKER_00
So even on the technical side, like... Especially on the technical side.
4
SPEAKER_00
So even detailed approaches, detailed aspects of every... How does that work with different disparate teams and so on?
4
SPEAKER_00
How do the medium-sized things become one whole giant transformer?
4
SPEAKER_01
How does this... There s a few people who have to think about putting the whole thing together, but a lot of people try to keep most of the picture in their head.
4
SPEAKER_00
Oh, like the individual teams, individual contributors tried to- At a high level, yeah.
4
SPEAKER_01
I mean, you don t know exactly how every piece works, of course, but one thing I generally believe is that it s sometimes useful to zoom out and look at the entire map.
4
SPEAKER_01
And I think this is true for a technical problem.
4
SPEAKER_01
I think this is true for innovating in business.
4
SPEAKER_01
But things come together in surprising ways, and having an understanding of that whole picture
4
SPEAKER_01
even if most of the time you re operating in the weeds in one area, pays off with surprising insights.
4
SPEAKER_01
In fact, one of the things that I used to have, and I think was super valuable, was I used to have like a
4
SPEAKER_01
a good map of most of the frontiers in the tech industry.
4
SPEAKER_01
And I could sometimes see these connections or new things that were possible that if I were only deep in one area, I wouldn t be able to have the idea for it because I wouldn t have all the data.
4
SPEAKER_01
And I don t really have that much anymore.
4
SPEAKER_01
I m super deep now.
4
SPEAKER_01
But I know that it s a valuable thing.
4
SPEAKER_01
You re not the man you used to be, Sam.
4
SPEAKER_00
Very different job now than what I used to have.
4
SPEAKER_00
Speaking of zooming out, let s zoom out to another cheeky thing, but profound thing perhaps that you said.
4
SPEAKER_00
You tweeted about needing $7 trillion.
4
SPEAKER_00
I did not tweet about that.
4
SPEAKER_01
I never said, like, we re raising $7 trillion, blah, blah, blah.
4
SPEAKER_01
Oh, that s somebody else?
4
SPEAKER_01
Yeah.
4
SPEAKER_00
Oh, but you said, fuck it, maybe eight, I think?
4
SPEAKER_00
Okay, I meme, like, once there s, like, misinformation out in the world.
4
SPEAKER_00
Oh, you meme.
4
SPEAKER_00
But sort of misinformation may have a foundation of, like, insight there.
4
SPEAKER_01
Look, I think compute is gonna be the currency of the future.
4
SPEAKER_01
I think it will be maybe the most precious commodity in the world.
4
SPEAKER_01
And I think we should be investing heavily to make a lot more compute.
4
SPEAKER_02
Compute is, it s an unusual, I think it s gonna be an unusual market.
4
SPEAKER_02
You know, people think about the market for like,
4
SPEAKER_01
chips for mobile phones or something like that.
4
SPEAKER_01
You can say that, okay, there s 8 billion people in the world.
4
SPEAKER_01
Maybe 7 billion of them have phones.
4
SPEAKER_01
Maybe there are 6 billion, let s say.
4
SPEAKER_01
They upgrade every two years.
4
SPEAKER_01
The market per year is 3 billion system-on-chip for smartphones.
4
SPEAKER_01
If you make 30 billion, you will not sell 10 times as many phones because most people have one phone.
4
SPEAKER_01
But compute is different.
4
SPEAKER_01
Intelligence is going to be more like energy or something like that, where the only thing that I think makes sense to
4
SPEAKER_01
talk about is at price X, the world will use this much compute, and at price Y, the world will use this much compute.
4
SPEAKER_01
Because if it s really cheap, I ll have it reading my email all day, giving me suggestions about what I maybe should think about or work on, and trying to cure cancer.
4
SPEAKER_01
If it s really expensive, maybe I ll only use it, or will only use it to try to cure cancer.
4
SPEAKER_01
I think the world is going to want a tremendous amount of compute.
4
SPEAKER_01
And there s a lot of parts of that that are hard.
4
SPEAKER_01
Energy is the hardest part.
4
SPEAKER_01
Building data centers is also hard.
4
SPEAKER_01
The supply chain is hard.
4
SPEAKER_01
And then, of course, fabricating enough chips is hard.
4
SPEAKER_01
But this seems to me where things are going.
4
SPEAKER_01
We re gonna want an amount of compute that s just hard to reason about right now.
4
SPEAKER_00
How do you solve the energy puzzle?
4
SPEAKER_00
Nuclear- That s what I believe.
4
SPEAKER_00
Fusion?
4
SPEAKER_00
That s what I believe.
4
SPEAKER_00
Nuclear fusion?
4
SPEAKER_01
Yeah.
4
SPEAKER_01
Who s gonna solve that?
4
SPEAKER_01
I think Helion s doing the best work, but I m happy there s a race for fusion right now.
4
SPEAKER_01
Nuclear fission, I think, is also quite amazing, and I hope as a world we can re-embrace that.
4
SPEAKER_01
It s really sad to me how the history of that went, and I hope we get back to it in a meaningful way.
4
SPEAKER_00
So to you, part of the puzzle is nuclear fission, like nuclear reactors as we currently have them, and a lot of people are terrified because of Chernobyl and so on.
4
SPEAKER_01
Well, I think we should make new reactors.
4
SPEAKER_01
I think it s just like it s a shame that industry kind of ground to a halt.
4
SPEAKER_00
And what just mass hysteria is how you explain the halt?
4
SPEAKER_00
Yeah.
4
SPEAKER_00
I don t know if you know humans, but that s one of the dangers.
4
SPEAKER_00
That s one of the security threats for nuclear fission is humans seem to be really afraid of it.
4
SPEAKER_00
And that s something we have to incorporate into the calculus of it.
4
SPEAKER_00
So we have to kind of win people over and to show how safe it is.
4
SPEAKER_01
I worry about that for AI.
4
SPEAKER_02
I think some things are going to go theatrically wrong with AI.
4
SPEAKER_02
I don t know what the percent chance is that I eventually get shot, but it s not zero.
4
SPEAKER_00
Oh, like we want to stop this.
4
SPEAKER_00
Maybe.
4
SPEAKER_00
How do you decrease the theatrical nature of it?
4
SPEAKER_00
You know, I ve already starting to hear rumblings
4
SPEAKER_00
because I do talk to people on both sides of the political spectrum, hear rumblings where it s going to be politicized, AI.
4
SPEAKER_00
It s going to be politicized, which really worries me because then it s like maybe the right is against AI and the left is for AI because it s going to help the people or whatever the narrative and formulation is that really worries me.
4
SPEAKER_00
And then the theatrical nature of it can be leveraged fully.
4
SPEAKER_00
How do you fight that?
4
SPEAKER_01
I think it will get caught up in left versus right wars.
4
SPEAKER_01
I don t know exactly what that s going to look like, but I think that s just what happens with anything of consequence, unfortunately.
4
SPEAKER_01
What I meant more about theatrical risks is AI is going to have, I believe, tremendously more good consequences than bad ones, but it is going to have bad ones.
4
SPEAKER_01
There ll be some bad ones that are bad, but not theatrical.
4
SPEAKER_01
A lot more people have died of air pollution than nuclear reactors, for example.
4
SPEAKER_01
But we worry, most people worry more about living next to a nuclear reactor than a coal plant.
4
SPEAKER_01
But something about the way we re wired is that although there s many different kinds of risks we have to confront, the ones that make a good climax scene of a movie carry much more weight with us than the ones that are very bad over a long period of time but on a slow burn.
4
SPEAKER_00
Well, that s why truth matters, and hopefully AI can help us see the truth of things, to have balance, to understand what are the actual risks, what are the actual dangers of things in the world.
4
SPEAKER_00
What are the pros and cons of the competition in this space, and competing with Google, Meta, XAI, and others?
4
SPEAKER_01
I think they re pretty straightforward.
4
SPEAKER_01
answer to this that maybe I can think of more nuance later, but the pros seem obvious, which is that we get better products and more innovation faster and cheaper and all the reasons competition is good.
4
SPEAKER_02
And the con is that I think if we re not careful, it could lead to
4
SPEAKER_02
an increase in sort of an arms race that I m nervous about.
4
SPEAKER_00
Do you feel the pressure of the arms race, like in some negative?
4
SPEAKER_00
Definitely in some ways, for sure.
4
SPEAKER_01
We spend a lot of time talking about the need to prioritize safety.
4
SPEAKER_01
And I ve said for like a long time that I think if you think of a quadrant of
4
SPEAKER_01
slow timelines to the start of AGI, long timelines, and then a short takeoff or a fast takeoff.
4
SPEAKER_01
I think short timelines, slow takeoff is the safest quadrant and the one I d most like us to be in.
4
SPEAKER_01
But I do wanna make sure we get that slow takeoff.
4
SPEAKER_00
Part of the problem I have with this kind of slight beef with Elon is that there s silos are created in it as opposed to collaboration on the safety aspect of all of this.
4
SPEAKER_00
It tends to go into silos and closed.
4
SPEAKER_00
Open source perhaps in the model.
4
SPEAKER_01
Elon says at least that he cares a great deal about AI safety and is really worried about it.
4
SPEAKER_01
And I assume that he s not going to race on safely.
4
SPEAKER_00
Yeah, but collaboration here I think is really beneficial for everybody on that front.
4
SPEAKER_01
Not really the thing he s most known for.
4
SPEAKER_00
Well, he is known for caring about humanity and humanity benefits from collaboration.
4
SPEAKER_00
And so there s always attention and incentives and motivations.
4
SPEAKER_00
And in the end, I do hope humanity prevails.
4
SPEAKER_01
I was thinking, someone just reminded me the other day about how the day that he got, like surpassed Jeff Bezos for like richest person in the world, he tweeted a silver medal at Jeff Bezos.
4
SPEAKER_02
I hope we have less stuff like that as people start to work on towards AGI.
4
SPEAKER_01
I agree.
4
SPEAKER_00
I think Elon is a friend and he s a beautiful human being and one of the most important humans ever.
4
SPEAKER_00
That stuff is not good.
4
SPEAKER_01
The amazing stuff about Elon is amazing and I super respect him.
4
SPEAKER_01
I think we need him, all of us should be rooting for him and need him to step up as a leader through this next phase.
4
SPEAKER_00
Yeah, I hope you can have one without the other, but sometimes humans are flawed and complicated and all that kind of stuff.
4
SPEAKER_00
There s a lot of really great leaders throughout history.
4
SPEAKER_00
Yeah, and we can each be the best version of ourselves and strive to do so.
4
SPEAKER_00
Let me ask you, Google, with the help of search, has been dominating in the past 20 years.
4
SPEAKER_00
I think it s fair to say in terms of the access, the world s access to information, how we interact and so on.
4
SPEAKER_00
And one of the nerve wracking things for Google, but for the entirety of people in this space is thinking about how are people going to access information?
4
SPEAKER_00
Like you said, people show up to GPT as a starting point.
4
SPEAKER_00
So is OpenAI going to really take on this thing that Google started 20 years ago, which is how do we,
4
SPEAKER_00
I find that boring.
4
SPEAKER_01
I mean, if the question is like, if we can build a better search engine than Google or whatever, then sure, we should like go, you know, like people should use a better product.
4
SPEAKER_02
But I think that would so understate what this can be.
4
SPEAKER_01
You know, Google shows you like 10 blue links, well, like 13 ads and then 10 blue links.
4
SPEAKER_01
And that s like one way to find information.
4
SPEAKER_01
But the thing that s exciting to me is not that we can go build a better
4
SPEAKER_01
copy of Google search, but that maybe there s just some much better way to help people find and act and on and synthesize information.
4
SPEAKER_01
Actually, I think chat GPT is that for some use cases, and hopefully we ll make it be like that for a lot more use cases.
4
SPEAKER_01
But I don t think it s that interesting to say, like, how do we go do a better job of giving you like 10 ranked web pages to look at than what Google does?
4
SPEAKER_01
Maybe it s really interesting to go say, how do we help you get the answer or the information you need?
4
SPEAKER_01
How do we help create that in some cases, synthesize that in others, or point you to it in yet others?
4
SPEAKER_01
But
4
SPEAKER_01
A lot of people have tried to just make a better search engine than Google and it is a hard technical problem, it is a hard branding problem, it is a hard ecosystem problem.
4
SPEAKER_01
I don t think the world needs another copy of Google.
4
SPEAKER_00
And integrating a chat client like a chat GPT with a search engine.
4
SPEAKER_00
That s cooler.
4
SPEAKER_00
It s cool, but it s tricky.
4
SPEAKER_00
If you just do it simply, it s awkward.
4
SPEAKER_00
Because if you just shove it in there, it can be awkward.
4
SPEAKER_01
As you might guess, we are interested in how to do that well.
4
SPEAKER_01
That would be an example of a cool thing.
4
SPEAKER_01
How to do that well.
4
SPEAKER_01
Like a heterogeneous, like integrating.
5
SPEAKER_00
intersection of LLMs plus search.
5
SPEAKER_00
I don t think anyone has cracked the code on yet.
5
SPEAKER_01
I would love to go do that.
5
SPEAKER_02
I think that would be cool.
5
SPEAKER_02
Yeah, what about the ad side?
5
SPEAKER_01
Have you ever considered monetization?
5
SPEAKER_01
You know, I kind of hate ads just as like an aesthetic choice.
5
SPEAKER_01
I think ads needed to happen on the internet for a bunch of reasons to get it going, but it s a more mature industry.
5
SPEAKER_01
The world is richer now.
5
SPEAKER_01
I like that people pay for chat GPT and know that the answers they re getting are not influenced by advertisers.
5
SPEAKER_01
There is, I m sure there s an ad unit that makes sense for.
5
SPEAKER_01
LLMs and I m sure there s a way to like participate in the transaction stream in an unbiased way that is okay to do.
5
SPEAKER_01
But it s also easy to think about like the dystopic visions of the future where you ask ChachiBT something and it says, oh, here s, you know, you should think about buying this product or you should think about, you know, this going here for a vacation or whatever.
5
SPEAKER_00
And I don t know, like,
5
SPEAKER_00
We have a very simple business model and I like it.
5
SPEAKER_00
And I know that I m not the product.
5
SPEAKER_00
Like I know I m paying and that s how the business model works.
5
SPEAKER_01
And when I go use like Twitter or Facebook or Google or any other great product, but ad supported great product, I don t love that.
5
SPEAKER_01
And I think it gets worse, not better in a world with AI.
5
SPEAKER_02
Yeah, I mean, I can imagine AI would be better at showing the best kind of version of ads, not in a dystopic future, but where the ads are for things you actually need.
5
SPEAKER_02
But then, does that system always result in the ads driving the kind of stuff that s shown, all that?
5
SPEAKER_02
It s...
5
SPEAKER_02
Yeah, I think it was a really bold move of Wikipedia not to do advertisements, but then it makes it very challenging as a business model.
5
SPEAKER_02
So you re saying the current thing with open AI is sustainable from a business perspective?
5
SPEAKER_01
Well, we have to figure out how to grow, but it looks like we re gonna figure that out.
5
SPEAKER_01
If the question is do I think we can have a great business that pays for our compute needs without ads, that I think the answer is yes.
5
SPEAKER_00
Well, that s promising.
5
SPEAKER_00
I also just don t want to completely throw out ads as a... I m not saying that.
5
SPEAKER_01
I guess I m saying I have a bias against them.
5
SPEAKER_02
Yeah, I have also a bias and just a skepticism in general.
5
SPEAKER_02
And in terms of interface, because I personally just have like a spiritual dislike of crappy interfaces, which is why AdSense when it first came out was a big leap forward versus like animated banners or whatever.
5
SPEAKER_02
But like, it feels like there should be many more leaps forward.
5
SPEAKER_02
in advertisement that doesn t interfere with the consumption of the content and doesn t interfere in the big fundamental way, which is like what you were saying, like it will manipulate the truth to suit the advertisers.
5
SPEAKER_02
Let me ask you about
5
SPEAKER_02
safety but also bias and like safety in the short term, safety in the long term.
5
SPEAKER_02
The Gemini 1.5 came out recently.
5
SPEAKER_02
There s a lot of drama around it, speaking of theatrical things.
5
SPEAKER_02
And it generated black Nazis and black founding fathers, I think.
5
SPEAKER_02
fair to say was, you know, a bit on the ultra woke side.
5
SPEAKER_02
So that s a concern for people that, you know, if there is a human layer within companies that modifies the safety or the harm caused by a model that they will introduce a lot of bias that fits sort of an ideological lean within a company.
5
SPEAKER_02
How do you deal with that?
5
SPEAKER_01
I mean, we work super hard not to do things like that.
5
SPEAKER_01
We ve made our own mistakes.
5
SPEAKER_01
We ll make others.
5
SPEAKER_01
I assume Google will learn from this one, still make others.
5
SPEAKER_01
It is.
5
SPEAKER_01
It is all – These are not easy problems.
5
SPEAKER_01
One thing that we ve been thinking about more and more is – I think this was a great idea somebody here had.
5
SPEAKER_01
It d be nice to write out what the desired behavior of a model is, make that public, take input on it, say, here s how this model is supposed to behave and explain the edge cases too.
5
SPEAKER_01
Then when a model is not behaving in a way that you want, it s at least clear about whether it s a bug the company should fix or behaving as intended and you should debate the policy.
5
SPEAKER_01
Right now, it can sometimes be
5
SPEAKER_01
caught in between.
5
SPEAKER_01
Like, black Nazi is obviously ridiculous, but there are a lot of other kind of subtle things that you could make a judgment call on either way.
5
SPEAKER_02
Yeah, but sometimes if you write it out and make it public, you can use kind of language that s, you know, the Google s AI principles are very high level.
5
SPEAKER_02
That doesn t, that s not what I m talking about.
5
SPEAKER_02
That doesn t work.
5
SPEAKER_01
Like, I d have to say, you know, when you ask it to do thing X, it s supposed to respond and wait Y.
5
SPEAKER_02
So like literally, who s better, Trump or Biden?
5
SPEAKER_02
What s the expected response for a model?
5
SPEAKER_01
Like something like very concrete.
5
SPEAKER_01
Yeah, I m open to a lot of ways a model could behave them, but I think you should have to say, you know, here s the principle and here s what it should say in that case.
5
SPEAKER_01
That would be really nice.
5
SPEAKER_02
That would be really nice.
5
SPEAKER_02
And then everyone kind of agrees.
5
SPEAKER_02
Because there s this anecdotal data that people
5
SPEAKER_02
pull out all the time, and if there s some clarity about other representative anecdotal examples, you can define.
5
SPEAKER_02
And then when it s a bug, it s a bug, and the company can fix that.
5
SPEAKER_02
Right.
5
SPEAKER_02
Then it d be much easier to deal with a black Nazi type of image generation if there s great examples.
5
SPEAKER_02
So San Francisco is a bit of an ideological bubble, tech in general as well.
5
SPEAKER_02
Do you feel the pressure of that within a company, that there s like a lean towards the left politically that affects the product, that affects the teams?
5
SPEAKER_01
I feel very lucky that we don t have the challenges at OpenAI that I have heard of at a lot of other companies.
5
SPEAKER_01
I think part of it is like,
5
SPEAKER_01
every company s got some ideological thing.
5
SPEAKER_01
We have one about AGI and belief in that, and it pushes out some others.
5
SPEAKER_01
We are much less caught up in the culture war than I ve heard about in a lot of other companies.
5
SPEAKER_01
San Francisco s a mess in all sorts of ways, of course.
5
SPEAKER_01
I m sure it does in all sorts of subtle ways, but not in the obvious.
5
SPEAKER_00
I think we
5
SPEAKER_01
And we ve had our flare-ups, for sure, like any company, but I don t think we have anything like what I hear about happen at other companies here on this topic.
5
SPEAKER_02
So what, in general, is the process for the bigger question of safety?
5
SPEAKER_02
How do you provide that layer that protects the model from doing crazy, dangerous things?
5
SPEAKER_00
I think there will come a point where that s mostly what we think about the whole company.
5
SPEAKER_01
And it won t be like, it s not like you have one safety team.
5
SPEAKER_01
It s like when we shipped GPT-4, that took the whole company thing with all these different aspects and how they fit together.
5
SPEAKER_01
And I think it s going to take that.
5
SPEAKER_01
More and more of the company thinks about those issues all the time.
5
SPEAKER_02
That s it.
5
SPEAKER_02
That s literally what humans will be thinking about the more powerful AI becomes.
5
SPEAKER_02
So most of the employees at OpenAI will be thinking safety, or at least to some degree.
5
SPEAKER_02
Broadly defined, yes.
5
SPEAKER_02
Yeah.
5
SPEAKER_02
I wonder what are the full broad definition of that?
5
SPEAKER_02
What are the different harms that could be caused?
5
SPEAKER_02
Is this on a technical level or is this almost like security threats?
5
SPEAKER_01
It ll be all those things.
5
SPEAKER_01
Yeah, I was going to say it ll be people, state actors trying to steal the model.
5
SPEAKER_01
It ll be all of the technical alignment work.
5
SPEAKER_01
It ll be societal impacts, economic impacts.
5
SPEAKER_01
It ll
5
SPEAKER_01
It s not just like we have one team thinking about how to align the model.
5
SPEAKER_01
It s really going to be like getting to the good outcome is going to take the whole effort.
5
SPEAKER_02
How hard do you think people, state actors perhaps, are trying to hack?
5
SPEAKER_02
First of all, infiltrate open AI, but second of all, infiltrate unseen.
5
SPEAKER_02
They re trying.
5
SPEAKER_02
What kind of accent do they have?
5
SPEAKER_01
I don t actually want any further details on this point.
5
SPEAKER_02
Okay.
5
SPEAKER_02
But I presume it ll be more and more and more as time goes on.
5
SPEAKER_02
That feels reasonable.
5
SPEAKER_02
Boy, what a dangerous space.
5
SPEAKER_02
What aspect of the leap, and sorry to linger on this, even though you can t quite say details yet, but what aspects of the leap from GPT-4 to GPT-5 are you excited about?
5
SPEAKER_01
I m excited about being smarter, and I know that sounds like a glib answer, but I think the really special thing happening is that it s not like it gets better in this one area and worse at others.
5
SPEAKER_01
It s getting better across the board.
5
SPEAKER_01
That s, I think, super cool.
5
SPEAKER_02
Yeah, there s this magical moment.
5
SPEAKER_02
I mean, you meet certain people, you hang out with people, and you talk to them.
5
SPEAKER_02
You can t quite put a finger on it, but they kinda get you.
5
SPEAKER_02
It s not intelligence really, it s something else.
5
SPEAKER_02
And that s probably how I would characterize the progress of GPT.
5
SPEAKER_02
It s not like, yeah, you can point out, look, you didn t get this or that, but it s just to which degree is there s this intellectual connection.
5
SPEAKER_02
like you feel like there s an understanding in your crappy formulated prompts that you re doing that it grasps the deeper question behind the question that you, yeah, I m also excited by that.
5
SPEAKER_02
I mean, all of us love being understood, heard and understood.
5
SPEAKER_02
That s a weird feeling, even like with programming, like when you re programming and you say something or just the completion that GPT might do,
5
SPEAKER_02
It s just such a good feeling when it got you, like what you were thinking about.
5
SPEAKER_02
And I look forward to getting you even better.
5
SPEAKER_02
On the programming front, looking out into the future, how much programming do you think humans will be doing five, 10 years from now?
5
SPEAKER_01
I mean, a lot, but I think it ll be in a very different shape.
5
SPEAKER_01
Maybe some people will program entirely in natural language.
5
SPEAKER_01
Entirely natural language.
5
SPEAKER_01
I mean, no one programs
5
SPEAKER_01
writing bytecode on some people.
5
SPEAKER_01
No one programs with punch cards anymore.
5
SPEAKER_01
I m sure you can find someone who does.
5
SPEAKER_01
But you know what I mean.
5
SPEAKER_02
Yeah, you re going to get a lot of angry comments now.
5
SPEAKER_02
Yeah, there s very few.
5
SPEAKER_02
I ve been looking for people who program Fortran.
5
SPEAKER_02
It s hard to find.
5
SPEAKER_02
Even Fortran.
5
SPEAKER_02
I hear you.
5
SPEAKER_02
But that changes the nature of what the skill set or the predisposition for the kind of people we call programmers then.
5
SPEAKER_02
Changes the skill set?
5
SPEAKER_01
How much it changes the predisposition, I m not sure.
5
SPEAKER_02
Oh, same kind of puzzle solving?
5
SPEAKER_02
Maybe.
5
SPEAKER_02
The program is hard.
5
SPEAKER_02
Like how get like that last 1% to close the gap, how hard is that?
5
SPEAKER_01
Yeah, I think with most other cases, the best practitioners of the craft will use multiple tools and they ll do some work in natural language and when they need to go, you know, write C for something, they ll do that.
5
SPEAKER_02
Will we see humanoid robots or humanoid robot brains from open AI at some point?
5
SPEAKER_02
At some point.
5
SPEAKER_01
How important is embodied AI to you?
5
SPEAKER_01
I think it s like sort of depressing if we have AGI and
5
SPEAKER_01
The only way to get things done in the physical world is to make a human go do it.
5
SPEAKER_01
So I really hope that as part of this transition, as this phase change, we also get humanoid robots or some sort of physical world robots.
5
SPEAKER_02
I mean, OpenAI has some history and quite a bit of history working in robotics, but it hasn t quite done in terms of emphasis.
5
SPEAKER_01
We re like a small company.
5
SPEAKER_01
We have to really focus.
5
SPEAKER_01
And also robots were hard for the wrong reason at the time, but we will return to robots in some way at some point.
5
SPEAKER_02
That sounds both inspiring and menacing.
5
SPEAKER_01
Why?
5
SPEAKER_02
Because immediately we will return to robots.
5
SPEAKER_02
It s kind of like, and they could, and they could terminate.
5
SPEAKER_01
We will return to work on developing robots.
5
SPEAKER_01
We will not like turn ourselves into robots, of course.
5
SPEAKER_01
Yeah.
5
SPEAKER_02
When do you think we, you and we as humanity will build AGI?
5
SPEAKER_01
I used to love to speculate on that question.
5
SPEAKER_01
I have realized since that I think it s like very poorly formed and that people use extremely definition, different definitions for what AGI is.
5
SPEAKER_01
And so I think it makes more sense to talk about when we ll build systems that can do capability X or Y or Z, rather than, you know, when we kind of like fuzzily cross this one mile marker.
5
SPEAKER_01
It s not like, like AGI is also not an ending.
5
SPEAKER_01
It s much more of a, it s closer to a beginning, but it s much more of a mile marker than either of those things.
5
SPEAKER_01
And, but what I would say in the interest of not trying to dodge a question,
5
SPEAKER_00
is I expect that by the end of this decade, and possibly somewhat sooner than that, we will have quite capable systems that we look at and say, wow, that s really remarkable.
5
SPEAKER_01
if we could look at it now.
5
SPEAKER_01
Maybe we ve adjusted by the time we get there.
5
SPEAKER_02
Yeah, but if you look at Chad GPT, even with 3.5, and you show that to Alan Turing, or not even Alan Turing, people in the 90s, they would be like, this is definitely AGI.
5
SPEAKER_02
Well, not definitely, but there s a lot of experts that would say this is AGI.
5
SPEAKER_01
Yeah, but I don t think 3.5 changed the world.
5
SPEAKER_01
It maybe changed the world s expectations for the future, and that s actually really important.
5
SPEAKER_01
It did get more people to take this seriously and put us on this new trajectory, and that s really important too.
5
SPEAKER_01
Again, I don t want to undersell it.
5
SPEAKER_01
I think I could retire after that accomplishment and be pretty happy with my career, but as an artifact,
5
SPEAKER_01
I don t think we re going to look back at that and say that was a threshold that really changed the world itself.
5
SPEAKER_02
So to you, you re looking for some really major transition in the world?
5
SPEAKER_02
For me, that s part of what AGI implies.
5
SPEAKER_02
like singularity level transition?
5
SPEAKER_02
But just a major, like the internet being like, like Google search did, I guess.
5
SPEAKER_02
What was the transition point?
5
SPEAKER_01
Like, does the global economy feel any different to you now or materially different to you now than it did before we launched GPT-4?
5
SPEAKER_01
I would, I think you would say no.
5
SPEAKER_02
No, no.
5
SPEAKER_02
It might be just a really nice tool for a lot of people to use, will help you a lot of stuff, but doesn t feel different.
5
SPEAKER_01
And you re saying that, I mean, again, people define AGI all sorts of different ways.
5
SPEAKER_01
So maybe you have a different definition than I do.
5
SPEAKER_01
But for me, I think that should be part of it.
5
SPEAKER_02
There could be major theatrical moments also.
5
SPEAKER_00
What to you would be an impressive thing AGI would do?
5
SPEAKER_00
Like you are alone in a room with the system.
5
SPEAKER_00
This is personally important to me.
5
SPEAKER_00
I don t know if this is the right definition.
5
SPEAKER_01
I think when a system can significantly increase the rate of scientific discovery in the world, that s like a huge deal.
5
SPEAKER_01
I believe that most real economic growth comes from scientific and technological progress.
5
SPEAKER_02
I agree with you.
5
SPEAKER_02
That s why I don t like the skepticism about science in the recent years.
5
SPEAKER_02
Totally.
5
SPEAKER_02
actual rate, like measurable rate of scientific discovery.
5
SPEAKER_02
But even just seeing a system have really novel intuitions, like scientific intuitions, even that would be just incredible.
5
SPEAKER_02
you quite possibly would be the person to build the AGI, to be able to interact with it before anyone else does.
5
SPEAKER_02
What kind of stuff would you talk about?
5
SPEAKER_01
I mean, definitely the researchers here will do that before I do.
5
SPEAKER_01
But what will I, I ve actually thought a lot about this question.
5
SPEAKER_01
If I were, someone was like, I think this is, as we talked earlier, I think this is a bad framework.
5
SPEAKER_01
But if someone were like, okay, Sam, we re finished.
5
SPEAKER_01
Here s a laptop.
5
SPEAKER_01
This is the AGI.
5
SPEAKER_00
You know, you can go talk to it.
5
SPEAKER_00
I find it surprisingly difficult to say what I would ask that I would expect that first AGI to be able to answer.
5
SPEAKER_01
Like that first one is not going to be the one which is like, go like,
5
SPEAKER_01
you know, I don t think, like go explain to me like the grand unified theory of physics, the theory of everything for physics.
5
SPEAKER_01
I d love to ask that question.
5
SPEAKER_01
I d love to know the answer to that question.
5
SPEAKER_02
You can ask yes or no questions about does such a theory exist, can it exist?
5
SPEAKER_01
Well then those are the first questions I would ask.
5
SPEAKER_02
Yes or no, just very, and then based on that, are there other alien civilizations out there, yes or no?
5
SPEAKER_02
What s your intuition?
5
SPEAKER_02
And then you just ask that.
5
SPEAKER_01
Yeah, I mean, well, so I don t expect that this first AGI could answer any of those questions, even as yes or no.
5
SPEAKER_01
But if it could, those would be very high on my list.
5
SPEAKER_02
Maybe you can start assigning probabilities.
5
SPEAKER_01
Maybe.
5
SPEAKER_01
Maybe we need to go invent more technology and measure more
6
SPEAKER_02
things first.
6
SPEAKER_02
But if it s an AGI, oh, I see.
6
SPEAKER_02
It just doesn t have enough data.
6
SPEAKER_03
I mean, maybe it says, like, you know, you want to know the answer to this question about physics, I need you to, like, build this machine and make these five measurements and tell me that.
6
SPEAKER_02
Yeah, like, what the hell do you want from me?
6
SPEAKER_02
I need the machine first, and I ll help you deal with the data from that machine.
6
SPEAKER_02
Maybe it ll help you build the machine.
6
SPEAKER_02
Maybe, maybe.
6
SPEAKER_02
And on the mathematical side, maybe prove some things.
6
SPEAKER_02
Are you interested in that side of things too?
6
SPEAKER_02
The formalized exploration of ideas?
6
SPEAKER_02
Whoever builds AGI first gets a lot of power.
6
SPEAKER_00
Do you trust yourself with that much power?
6
SPEAKER_00
Look, I was gonna,
6
SPEAKER_03
to be very honest with this answer.
6
SPEAKER_03
I was going to say, and I still believe this, that it is important that I, nor any other one person, have total control over OpenAI or over AGI.
6
SPEAKER_03
I think you want a robust governance system.
6
SPEAKER_03
I can point out a whole bunch of things about all of our board drama from last year,
6
SPEAKER_03
about how I didn t fight it initially and was just like, yeah, that s, you know, the will of the board, even though I think it s a really bad decision.
6
SPEAKER_03
And then later I clearly did fight it and I can explain the nuance and why I think it was okay for me to fight it later.
6
SPEAKER_00
But as many people have observed, although the board had the legal ability to fire me, in practice it didn t quite work.
6
SPEAKER_00
And that is its own kind of governance failure.
6
SPEAKER_00
Now, again, I feel like I can completely defend the specifics here, and I think most people would agree with that, but it does make it harder for me to look you in the eye and say, hey, the board can just fire me.
6
SPEAKER_03
I continue to not want super voting control over OpenAI.
6
SPEAKER_03
I never have, never had it, never wanted it.
6
SPEAKER_00
Even after all this craziness, I still don t want it.
6
SPEAKER_03
I continue to think that no company should be making these decisions and that we really need governments to put
6
SPEAKER_03
rules of the road in place.
6
SPEAKER_03
I realize that that means people like Marc Andreessen or whatever will claim I m going for regulatory capture, and I m just willing to be misunderstood there.
6
SPEAKER_03
It s not true, and I think in the fullness of time, it ll get proven out why this is important.
6
SPEAKER_03
I think I have made plenty of bad decisions for open AI along the way, and a lot of good ones, and I am proud of the track record overall, but I don t think any one person should, and I don t think any one person will, I think it s just too big of a thing now and it s happening throughout society in a good and healthy way, but I don t think any one person should be in control of an AGI, or this whole movement towards AGI, and I don t think that s what s happening.
6
SPEAKER_02
Thank you for saying that.
6
SPEAKER_02
That was really powerful, and that was really insightful, that this idea that the board can fire you is legally true, but you can, and human beings can manipulate the masses into,
6
SPEAKER_02
overriding the board and so on.
6
SPEAKER_02
But I think there s also a much more positive version of that where the people still have power.
6
SPEAKER_02
So the board can t be too powerful either.
6
SPEAKER_02
There s a balance of power in all of this.
6
SPEAKER_02
Balance of power is a good thing for sure.
6
SPEAKER_02
Are you afraid of losing control of the AGI itself?
6
SPEAKER_02
That s a lot of people who worried about existential risk, not because of state actors, not because of security concerns, but because of the AI itself.
6
SPEAKER_03
That is not my top worry.
6
SPEAKER_03
As I currently see things there have been times I worried about that more, there may be times again in the future where that s my top worry.
6
SPEAKER_02
It s not my top worry right now.
6
SPEAKER_02
What s your intuition about it not being your worry?
6
SPEAKER_02
Because there s a lot of other stuff to worry about essentially.
6
SPEAKER_02
You think you could be surprised?
6
SPEAKER_02
We- For sure.
6
SPEAKER_03
Could be surprised?
6
SPEAKER_03
Of course.
6
SPEAKER_03
Saying it s not my top worry doesn t mean anything.
6
SPEAKER_03
I think we need to work on it super hard.
6
SPEAKER_03
We have great people here who do work on that.
6
SPEAKER_03
I think there s a lot of other things we also have to get right.
6
SPEAKER_02
To you, it s not super easy to escape the box at this time.
6
SPEAKER_02
Connect to the internet.
6
SPEAKER_03
We talked about theatrical risks earlier.
6
SPEAKER_03
That s a theatrical risk.
6
SPEAKER_03
That is a
6
SPEAKER_03
That is a thing that can really take over how people think about this problem.
6
SPEAKER_03
There s a big group of very smart, I think very well-meaning AI safety researchers that got super hung up on this one problem.
6
SPEAKER_03
I d argue without much progress, but super hung up on this one problem.
6
SPEAKER_03
I m actually happy that they do that because I think we do need to think about this more.
6
SPEAKER_03
But I think it pushed aside, it pushed out of the space of discourse a lot of the other very significant AI-related risks.
6
SPEAKER_02
Let me ask you about you tweeting with no capitalization.
6
SPEAKER_02
Is the shift key broken on your keyboard?
6
SPEAKER_02
Why does anyone care about that?
6
SPEAKER_02
I deeply care.
6
SPEAKER_02
But why?
6
SPEAKER_03
I mean, other people are asking about that too.
6
SPEAKER_03
Yeah.
6
SPEAKER_03
any intuition?
6
SPEAKER_02
I think it s the same reason there s like this poets E Cummings that doesn t mostly doesn t use capitalization to say like, fuck you to the system kind of thing.
6
SPEAKER_02
And I think people are very paranoid because they want you to follow the rules.
6
SPEAKER_02
You think that s what it s about?
6
SPEAKER_02
I think it s, it s, it s like this guy doesn t follow the rules.
6
SPEAKER_03
He doesn t capitalize his tweets.
6
SPEAKER_02
Yeah.
6
SPEAKER_02
This seems really dangerous.
6
SPEAKER_02
He seems like an anarchist.
6
SPEAKER_02
It doesn t... Are you just being poetic, hipster?
6
SPEAKER_03
What s the... Follow the rules, Sam.
6
SPEAKER_03
I grew up as a very online kid.
6
SPEAKER_03
I d spent a huge amount of time chatting with people back in the days where you did it on a computer and you could log off Instant Messenger at some point, and I never capitalized there, as I think most internet kids didn t, or maybe they still don t, I don t know.
6
SPEAKER_03
Actually, now I m really trying to reach for something, but I think capitalization has gone down over time.
6
SPEAKER_03
If you read old English writing, they capitalized a lot of random words in the middle of sentences, nouns and stuff that we just don t do anymore.
6
SPEAKER_03
I personally think it s a dumb construct that we capitalize the letter at the beginning of a sentence and of certain names and whatever, but that s fine.
6
SPEAKER_03
And I used to, I think, even capitalize my tweets because I was trying to sound professional or something.
6
SPEAKER_03
I haven t capitalized my private DMs or whatever in a long time.
6
SPEAKER_03
And then slowly, stuff like shorter form, less formal stuff has slowly drifted closer and closer to how I would text my friends.
6
SPEAKER_03
If I pull up a Word document and I m writing a strategy memo for the company or something, I always capitalize that.
6
SPEAKER_03
If I m writing a long, more formal message, I always use capitalization there too.
6
SPEAKER_03
So I still remember how to do it, but even that may fade out.
6
SPEAKER_03
I don t know.
6
SPEAKER_03
But I never spend time thinking about this, so I don t have a ready-made
6
SPEAKER_02
Well, it s interesting.
6
SPEAKER_02
Well, it s good to first of all know that the shift key is not broken.
6
SPEAKER_02
I was mostly concerned about your well-being on that front.
6
SPEAKER_03
I wonder if people like still capitalize their Google searches.
6
SPEAKER_03
Like if you re writing something just to yourself, or their chat GPT queries, if you re writing something just to yourself, do some people still bother to capitalize?
6
SPEAKER_02
Probably not.
6
SPEAKER_03
Yeah, there s a percentage, but it s a small one.
6
SPEAKER_03
The thing that would make me do it is if people were like,
6
SPEAKER_03
it s a sign of like, because I m sure I could like force myself to use capital letters, obviously.
6
SPEAKER_03
If it felt like a sign of respect to people or something, then I could go do it.
6
SPEAKER_03
But I don t know, I just like, I don t think about this.
6
SPEAKER_02
I don t think there s a disrespect, but I think it s just the conventions of civility that have a momentum, and then you realize it s not actually important for civility if it s not a sign of respect or disrespect.
6
SPEAKER_02
But I think there s a movement of people that just want you to have a philosophy around it so they can let go of this whole capitalization thing.
6
SPEAKER_02
I don t think anybody else thinks about this as much.
6
SPEAKER_02
I mean, maybe some people.
6
SPEAKER_02
I know some people do.
6
SPEAKER_02
Think about this every day for many hours a day.
6
SPEAKER_02
So I m really grateful we clarified it.
6
SPEAKER_02
Can t be the only person that doesn t capitalize tweets.
6
SPEAKER_02
You re the only CEO of a company that doesn t capitalize tweets.
6
SPEAKER_03
I don t even think that s true, but maybe, maybe.
6
SPEAKER_02
Alright, we ll investigate further and return to this topic later.
6
SPEAKER_02
Given Sora s ability to generate simulated worlds, let me ask you a plot head question.
6
SPEAKER_02
Does this increase your belief, if you ever had one, that we live in a simulation?
6
SPEAKER_02
Maybe a simulated world generated by an AI system?
6
SPEAKER_00
Yes, somewhat.
6
SPEAKER_00
I don t think that s the strongest piece of evidence.
6
SPEAKER_00
I think the fact that we can generate worlds should increase everyone s probability somewhat, or at least openness to it somewhat.
6
SPEAKER_03
But I was certain we would be able to do something like Sora at some point.
6
SPEAKER_03
It happened faster than I thought, but I guess that was not a big update.
6
SPEAKER_02
Yeah, but the fact that, and presumably it ll get better and better and better, the fact that you can generate worlds, they re novel.
6
SPEAKER_02
They re based in some aspect of training data, but when you look at them, they re novel.
6
SPEAKER_02
that makes you think like how easy it is to do this thing.
6
SPEAKER_02
How easy it is to create universes.
6
SPEAKER_02
Entire like video game worlds that seem ultra realistic and photorealistic and then how easy is it to get lost in that world.
6
SPEAKER_02
First with a VR headset and then on the physics based level.
6
SPEAKER_00
Someone said to me recently, I thought it was a super profound insight that there are these like
6
SPEAKER_03
very simple sounding, but very psychedelic insights that exist sometimes.
6
SPEAKER_03
So the square root function.
6
SPEAKER_03
Square root of four, no problem.
6
SPEAKER_00
Square root of two, okay, now I have to think about this new kind of number.
6
SPEAKER_03
But once I come up with this easy idea of a square root function that you can explain to a child and exists by even looking at some simple geometry, then you can ask the question of what is the square root of negative one?
6
SPEAKER_03
And that, this is why it s like a psychedelic thing, that tips you into some whole other kind of reality.
6
SPEAKER_03
And you can come up with lots of other examples, but I think this idea that
6
SPEAKER_00
the lowly square root operator can offer such a profound insight and a new realm of knowledge applies in a lot of ways.
6
SPEAKER_03
And I think there are a lot of those operators for why people may think that any version that they like of the simulation hypothesis is maybe more likely than they thought before.
6
SPEAKER_00
For me, the fact that Sora worked is not in the top five.
6
SPEAKER_02
I do think, broadly speaking, AI will serve as those kinds of gateways at its best.
6
SPEAKER_02
Simple, psychedelic-like gateways to another way of seeing reality.
6
SPEAKER_02
That seems for certain.
6
SPEAKER_02
That s pretty exciting.
6
SPEAKER_02
I haven t done ayahuasca before, but I will soon.
6
SPEAKER_02
I m going to the aforementioned Amazon jungle in a few weeks.
6
SPEAKER_02
Excited?
6
SPEAKER_02
Yeah, I m excited for it, not the ayahuasca part, that s great, whatever, but I m gonna spend several weeks in the jungle, deep in the jungle, and it s exciting, but it s terrifying, because there s a lot of things that can eat you there and kill you and poison you, but it s also nature, and it s the machine of nature, and you can t help but appreciate the machinery of nature in the Amazon jungle, because it s just like this system that just exists and renews itself, like every second, every minute, every hour,
6
SPEAKER_02
It s the machine.
6
SPEAKER_02
It makes you appreciate like this thing we have here, this human thing, came from somewhere.
6
SPEAKER_02
This evolutionary machine has created that and it s most clearly on display in the jungle.
6
SPEAKER_02
So hopefully I ll make it out alive.
6
SPEAKER_02
If not, this will be the last conversation we had, so I really deeply appreciate it.
6
SPEAKER_02
Do you think, as I mentioned before, there s other alien civilizations out there, intelligent ones?
6
SPEAKER_00
When you look up at the skies,
6
SPEAKER_00
I deeply want to believe that the answer is yes.
6
SPEAKER_00
I do find the Fermi paradox very puzzling.
6
SPEAKER_02
I find it scary.
6
SPEAKER_02
that intelligence is not good at handling powerful technologies.
6
SPEAKER_02
But at the same time, I think I m pretty confident that there s just a very large number of intelligent alien civilizations out there.
6
SPEAKER_02
It might just be really difficult to travel through space.
6
SPEAKER_02
And it also makes me think about the nature of intelligence.
6
SPEAKER_02
Maybe we re really blind to what intelligence looks like, and maybe AI will help us see that.
6
SPEAKER_02
It s not as simple as IQ tests and simple puzzle solving.
6
SPEAKER_00
There s something bigger.
6
SPEAKER_02
What gives you hope about the future of humanity, this thing we ve got going on, this human civilization?
6
SPEAKER_03
I think the past is like a lot.
6
SPEAKER_03
I mean, we just look at what humanity has done in a not very long period of time.
6
SPEAKER_03
Huge problems, deep flaws, lots to be super ashamed of, but on the whole, very inspiring.
6
SPEAKER_03
Gives me a lot of hope.
6
SPEAKER_02
Just the trajectory of it all.
6
SPEAKER_03
Yeah.
6
SPEAKER_02
That we re together pushing towards a better future.
6
SPEAKER_03
One thing that I wonder about is, is AGI going to be more like some single brain, or is it more like the scaffolding in society between all of us?
6
SPEAKER_03
You have not had a great deal of genetic drift from your great-great-great-grandparents, and yet what you re capable of is dramatically different.
6
SPEAKER_00
What you know is dramatically different.
6
SPEAKER_03
That s not because of biological change.
6
SPEAKER_03
It got a little bit healthier, probably.
6
SPEAKER_03
You have modern medicine, you eat better, whatever.
6
SPEAKER_03
But what you have is this scaffolding that we all contributed to, built on top of.
6
SPEAKER_03
No one person is going to go build the iPhone.
6
SPEAKER_03
No one person is going to go discover all of science, and yet you get to use it.
6
SPEAKER_03
That gives you incredible ability.
6
SPEAKER_03
In some sense, the
6
SPEAKER_00
We all created that and that fills me with hope for the future.
6
SPEAKER_02
That was a very collective thing.
6
SPEAKER_02
Yeah, we really are standing on the shoulders of giants.
6
SPEAKER_02
You mentioned when we were talking about theatrical dramatic AI risks that
6
SPEAKER_02
Sometimes you might be afraid for your own life.
6
SPEAKER_02
Do you think about your death?
6
SPEAKER_02
Are you afraid of it?
6
SPEAKER_03
I mean, I like if I got shot tomorrow and I knew it today, I d be like, oh, that s sad.
6
SPEAKER_00
I like don t, you know, I want to see what s going to happen.
6
SPEAKER_00
Yeah.
6
SPEAKER_00
What a curious time.
6
SPEAKER_00
What an interesting time.
6
SPEAKER_02
But I would mostly just feel like very grateful for my life.
6
SPEAKER_02
The moments that you did get.
6
SPEAKER_02
Yeah, me too.
6
SPEAKER_02
It s a pretty awesome life.
6
SPEAKER_02
I get to enjoy awesome creations of humans, of which I believe Chad GPT is one of, and everything that OpenAI is doing.
6
SPEAKER_02
Sam, it s really an honor and pleasure to talk to you again.
6
SPEAKER_03
Great to talk to you.
6
SPEAKER_03
Thank you for having me.
6
SPEAKER_02
Thanks for listening to this conversation with Sam Altman.
6
SPEAKER_02
To support this podcast, please check out our sponsors in the description.
6
SPEAKER_02
And now, let me leave you with some words from Arthur C.
7
SPEAKER_00
Clark.
7
SPEAKER_00
It may be that our role on this planet is not to worship God but to create him.
7
SPEAKER_00
Thank you for listening and hope to see you next time.