0
SPEAKER_02
The Joe Rogan Experience.
0
SPEAKER_01
Oh, you know, not too much.
0
SPEAKER_00
Just another typical week in AI.
0
SPEAKER_00
Just the beginning of the end of time.
0
SPEAKER_00
That s all happening right now.
0
SPEAKER_00
Just for the sake of the listeners, please just give us your names and tell us what you do.
0
SPEAKER_01
So, I m Jeremy Harris.
0
SPEAKER_01
I m the CEO and co-founder of this company, Gladstone AI, that we co-founded.
0
SPEAKER_01
So, we re essentially a national security and AI company.
0
SPEAKER_01
We can get into the backstory a little bit later, but that s the high level.
0
SPEAKER_02
I m Ed Harris.
0
SPEAKER_02
I m actually I m his co-founder and brother and the CTO of the company Keep this like pull this up like a fist from your face.
0
SPEAKER_00
There you go.
0
SPEAKER_00
Perfect.
0
SPEAKER_00
So How long have you guys been involved in the whole AI space?
0
SPEAKER_01
for a while in different ways.
0
SPEAKER_01
We started off as physicists.
0
SPEAKER_01
That was our background.
0
SPEAKER_01
Around 2017, we started to go into AI startups.
0
SPEAKER_01
We founded a startup, took it through Y Combinator, this Silicon Valley accelerator program.
0
SPEAKER_01
At the time, actually, Sam Altman, who s now the CEO of OpenAI, was the president of Y Combinator.
0
SPEAKER_01
He opened up our batch at YC with this big speech, and we got some conversations in with him over the course of the batch.
0
SPEAKER_01
In 2020, so this thing happened that we can talk about, essentially this was like the moment that there s like a before and after in the world of AI, before and after 2020.
0
SPEAKER_01
And it launched this revolution that brought us to ChatGPT.
0
SPEAKER_01
Essentially there was an insight that OpenAI had and doubled down on that you can draw a straight line to ChatGPT, GPT-4, Google Gemini, everything that makes AI everything it is today started then.
0
SPEAKER_01
And when it happened, we kind of went, well, Ed gave me a call, this panicked phone call.
0
SPEAKER_01
He s like, dude, I don t think we can keep working like business as usual in a company.
0
SPEAKER_02
In a regular company anymore, yeah.
0
SPEAKER_02
Yeah.
0
SPEAKER_02
So there was this AI model called GPT-3.
0
SPEAKER_02
So everyone has maybe played with GPT-4, that s like chat GPT.
0
SPEAKER_02
GPT-3 was the generation before that.
0
SPEAKER_02
And it was the first time that you had an AI model that could,
0
SPEAKER_02
that could actually, let s say, do stuff like write news articles that the average person, like in a paragraph of a news article, could not tell the difference between it wrote this news article and a real person wrote this news article.
0
SPEAKER_02
So that was an inflection, and that was significant in itself.
0
SPEAKER_02
But what was most significant was that it represented a point along this line, this scaling trend for AI,
0
SPEAKER_02
where the signs were that you didn t have to be clever.
0
SPEAKER_02
You didn t have to come up with necessarily a revolutionary new algorithm or be smart about it.
0
SPEAKER_02
You just had to take what works and make it way, way, way bigger.
0
SPEAKER_02
And the significance of that is you increase the amount of computing cycles you put against something, you increase the amount of data.
0
SPEAKER_02
All of that is an engineering problem, and you can solve it with money.
0
SPEAKER_02
So you can scale up the system, use it to make money, and put that money right back into scaling up the system some more.
0
SPEAKER_02
Money in, IQ points come out.
0
SPEAKER_01
That was kind of the 2020 moment.
0
SPEAKER_01
And that s what we said in 2020, exactly.
0
SPEAKER_01
I spent about two hours trying to argue him out of it.
0
SPEAKER_01
I was like, no, no, no, we can keep working at our company, because we re having fun.
0
SPEAKER_01
We like founding companies.
0
SPEAKER_01
And yeah, he just wrestled me to the ground.
0
SPEAKER_01
And we re like, shit, we ve got to do something about this.
0
SPEAKER_01
We reached out to a family friend who, he was non-technical, but he had some connections in government in DOD.
0
SPEAKER_01
And we re like, dude, the way this is set up right now,
0
SPEAKER_01
You can really start drawing straight lines and extrapolating and saying, you know what, the government is going to give a shit about this in.
0
SPEAKER_01
Not very long, two years, four years, we re not sure, but the knowledge about what s going on here is so siloed in the Frontier Labs.
0
SPEAKER_01
Our friends are all over the Frontier Labs, the Open AIs, the Google DeepMinds, all that stuff.
0
SPEAKER_01
The shit they were saying to us that was like mundane reality, like water cooler conversation, when you then went to talk to people in policy and even pretty senior people in government,
0
SPEAKER_01
not tracking the story remotely.
0
SPEAKER_01
In fact, you re hearing almost a diametric opposite.
0
SPEAKER_01
This is like over-learning the lessons of the AI winters that came before when it s pretty clear like we re on a very, at least, interesting trajectory, let s say, that should change the way we re thinking about the technology.
0
SPEAKER_00
What was your fear?
0
SPEAKER_00
Like what was it that hit you that made you go, we have to stop doing this?
0
SPEAKER_02
So it s basically, you know, anyone can draw a straight line, right, on a graph.
0
SPEAKER_02
The key is looking ahead and actually at that point, three years out, four years out, and asking like you re asking, what does this mean for the world?
0
SPEAKER_02
What does it mean?
0
SPEAKER_02
What does the world have to look like if we re at this point?
0
SPEAKER_02
we re already seeing the first kind of wave of risk sets just begin to materialize, and that s kind of the weaponization risk sets.
0
SPEAKER_02
So you think about stuff like large-scale psychological manipulation of social media, actually really easy to do now.
0
SPEAKER_02
You train a model on just a whole bunch of tweets.
0
SPEAKER_02
You can actually direct it to push a narrative like, you know, maybe China should own Taiwan or, you know, whatever, something like that.
0
SPEAKER_02
And you actually – you can train it to adjust the discourse and have increasing levels of effectiveness to that.
0
SPEAKER_02
Just as you increase the general capability surface of these systems, we don t know how to predict what exactly comes out of them at each level of scale.
0
SPEAKER_02
But it s just general increasing power.
0
SPEAKER_02
And then the kind of next beat of risk after that.
0
SPEAKER_02
So we re scaling these systems, we re on track to scale systems that are at human level, like generally as smart, however you define that, as a person or greater.
0
SPEAKER_02
And OpenAI and the other labs are saying, yeah, it might be two years away, three years away, four years away, like insanely close.
0
SPEAKER_02
At the same time, and we can go into the details of this, but we actually don t understand how to reliably control these systems.
0
SPEAKER_02
We don t understand how to get these systems to do
0
SPEAKER_02
what it is we want.
0
SPEAKER_02
We can kind of like poke them and prod them and get them to kind of adjust.
0
SPEAKER_02
But you ve seen, and we can go over these examples, we ve seen example after example of, you know, Bing Sidney yelling at users, Google showing 17th century British scientists that are racially diverse, all that kind of stuff.
0
SPEAKER_02
We don t really understand how to like aim it or align it or steer it.
0
SPEAKER_02
And so then you can ask yourself, well, we re on track to get here.
0
SPEAKER_02
we are not on track to control these systems effectively.
0
SPEAKER_02
How bad is that?
0
SPEAKER_02
And the risk is if you have a system that is significantly smarter than humans or human organization, that we basically get disempowered in various ways relative to that system.
0
SPEAKER_02
And we can go into some details on that too.
0
SPEAKER_00
Now, when a system does something like what Gemini did, like it says, show us Nazi soldiers and it shows you Asian women, what s the mechanism?
0
SPEAKER_00
How does that happen?
0
SPEAKER_01
It s maybe worth taking a step back and looking at how these systems actually work.
0
SPEAKER_01
That s going to give us a bit of a frame too for figuring out when we see weird shit happen
0
SPEAKER_01
How weird is that shit?
0
SPEAKER_01
Is that shit just explainable by just the basic mechanics of what you would expect to happen based on the way we re training these things?
0
SPEAKER_01
Or is something new and fundamentally different happening?
0
SPEAKER_01
So we re talking about this idea of scaling these AI systems, right?
0
SPEAKER_01
What does that actually mean?
0
SPEAKER_01
Well, you imagine the AI model, which is kind of like you think of it as like the artificial brain here that actually does the thinking.
0
SPEAKER_01
That model contains, it s kind of like a human brain.
0
SPEAKER_01
It s got these things called neurons.
0
SPEAKER_01
We, in the human brain, call them biological neurons.
0
SPEAKER_01
In the context of AI, it s artificial neurons, but it doesn t really matter.
0
SPEAKER_01
They re the cells that do the thinking for the machine.
0
SPEAKER_01
And the realization of AI scaling is that you can basically take this model, increase the number of artificial neurons it contains.
0
SPEAKER_01
And at the same time, increase the amount of computing power that you re putting into kind of like wiring the connections between those neurons.
0
SPEAKER_01
That s the training process.
0
SPEAKER_00
Can I pause you right there?
0
SPEAKER_01
Yeah.
0
SPEAKER_01
How does the neuron think?
0
SPEAKER_01
Yeah, so okay.
0
SPEAKER_01
So let s get a little bit more concrete then.
0
SPEAKER_01
So in your brain, right, we have these neurons.
0
SPEAKER_01
They re all connected to each other with different connections.
0
SPEAKER_01
And when you go out into the world and you learn a new skill, what really happens is you try out that skill, you succeed or fail, and based on your succeeding or failing, the connections between neurons that are associated with doing that task well get stronger.
0
SPEAKER_01
The connections that are associated with doing it badly get weaker.
0
SPEAKER_01
And over time, through this glorified process of trial and error, eventually you re going to hone in and really, in a very real sense, everything you know about the world gets implicitly encoded in the strengths of the connections.
1
SPEAKER_00
between all those neurons.
1
SPEAKER_00
If I can x-ray your brain and get all the connection strengths of all the neurons, I have everything Joe Rogan has learned about the world.
1
SPEAKER_00
That s like basically a good sketch, let s say, of what s going on here.
1
SPEAKER_00
So now we apply that to AI, right?
1
SPEAKER_00
That s the next step.
1
SPEAKER_00
And here, really, it s the same story.
1
SPEAKER_00
We have these massive systems, artificial neurons connected to each other.
1
SPEAKER_00
The strength of those connections is secretly what encodes all the knowledge.
1
SPEAKER_00
So if I can steal
1
SPEAKER_00
All of those connections, those weights, as they re sometimes called, I ve stolen the model.
1
SPEAKER_00
I ve stolen the artificial brain.
1
SPEAKER_00
I can use it to do whatever the model could do initially.
1
SPEAKER_00
That is kind of the artifact of central interest here.
1
SPEAKER_00
And so if you can build the system right, now you ve got so many moving parts.
1
SPEAKER_00
Like if you look at GPT-4, it has, people think, around a trillion of these connections.
1
SPEAKER_00
And that s a trillion little pieces that all have to be jiggered together to work together coherently.
1
SPEAKER_00
And you need computers to go through and tweak those numbers.
1
SPEAKER_00
So massive amounts of computing power.
1
SPEAKER_00
The bigger you make that model, the more computing power you re going to need to tune it in.
1
SPEAKER_00
And now you have this relationship between the size of your model, the amount of computing power you re going to use to train it.
1
SPEAKER_00
And if you can increase those things at the same time, what Ed was saying is your IQ points basically drop out.
1
SPEAKER_00
Very roughly speaking, that was what people realized in 2020.
1
SPEAKER_00
And the effect that had, right, was now all of a sudden the entire AI industry is looking at this equation.
1
SPEAKER_00
Everybody knows the secret sauce.
1
SPEAKER_00
I make it bigger, I make more IQ points, I can get more money.
1
SPEAKER_00
So Google s looking at this, Microsoft, OpenAI, Amazon, everybody s looking at the same equation.
1
SPEAKER_00
You have the makings for a crazy race.
1
SPEAKER_00
Right now today, Microsoft is engaged in the single biggest infrastructure in human history
1
SPEAKER_00
build out the biggest infrastructure, build out $50 billion a year.
1
SPEAKER_00
Right.
1
SPEAKER_00
So on the scale of the Apollo moon landings, just in building out data centers to house the compute infrastructure, because they are betting that these systems are going to get them to something like human level AI pretty damn soon.
1
SPEAKER_02
Oh, so I was reading some story about I think it was Google that saying that they re going to have multiple nuclear reactors to power their their database.
1
SPEAKER_01
That s the that s what you got to do now because what s going on is North America is kind of running out of
1
SPEAKER_01
on-grid baseload power to actually supply these data centers.
1
SPEAKER_01
You re getting data center building moratoriums in areas like Virginia, which has traditionally been the data center cluster for Amazon, for example, and for a lot of these other companies.
1
SPEAKER_01
And so, when you build a data center, you need a bunch of resources, you know, cited close to that data center.
1
SPEAKER_01
You need water for cooling and a source of electricity.
1
SPEAKER_01
And it turns out that, you know, wind and solar don t really quite cut it for these big data centers that train big models, because the data center, the training consumes power like this all the time, but the sun isn t always shining, the wind isn t always blowing.
1
SPEAKER_01
And so,
1
SPEAKER_01
You ve got to build nuclear reactors, which give you high capacity factor base load.
1
SPEAKER_01
And Amazon literally bought, yeah, a data center with a nuclear plant right next to it, because that s what you ve got to do.
1
SPEAKER_01
Jesus.
1
SPEAKER_02
How long does it take to build a nuclear reactor?
1
SPEAKER_02
Because this is the race, right?
1
SPEAKER_02
The race is, you re talking about 2020, people realizing this, then you have to have the power to supply it.
1
SPEAKER_02
But how long, how many years does it take to get an active nuclear reactor up and running?
1
SPEAKER_01
It s an answer that depends.
1
SPEAKER_01
The Chinese are faster than us at building nuclear reactors, for example.
1
SPEAKER_00
And that s part of the geopolitics of this too, right?
1
SPEAKER_00
Yeah.
1
SPEAKER_00
When you look at US versus China,
1
SPEAKER_00
What is bottlenecking each country?
1
SPEAKER_00
Right.
1
SPEAKER_00
So the U.S.
1
SPEAKER_00
is bottlenecked increasingly by power, baseload power.
1
SPEAKER_00
China, because we ve got export control measures in place, in part as a response to the scaling phenomenon that as a result of the investigation we did.
1
SPEAKER_00
That s right, yeah.
1
SPEAKER_00
In part.
1
SPEAKER_00
In part, yeah.
1
SPEAKER_00
But China is bottlenecked by their access to the actual processors.
1
SPEAKER_00
They ve got all the power they can eat, because they ve got much more infrastructure investment, but the chip side is weaker.
1
SPEAKER_00
So there s sort of a balancing act between the two sides, and it s not clear yet which one positions you strategically for dominance in the long term.
1
SPEAKER_01
But we are also building better
1
SPEAKER_01
So, small modular reactors, essentially small nuclear power plants that can be mass produced.
1
SPEAKER_01
Those are starting to come online relatively early, but the technology and designs are pretty mature.
1
SPEAKER_01
So, that s probably the next beat for our power grid for data centers, I would imagine.
1
SPEAKER_01
Microsoft is doing this.
1
SPEAKER_02
So, in 2020, you have this revelation.
1
SPEAKER_02
You recognize where this is going.
1
SPEAKER_02
You see how it charts, and you say, this is going to be a real problem.
1
SPEAKER_02
Does anybody listen to you?
1
SPEAKER_01
This is where the problem comes, right?
1
SPEAKER_01
Yeah, like we said, right?
1
SPEAKER_01
You can draw a straight line, you can have people nodding along, but there s a couple of hiccups along the way.
1
SPEAKER_01
One, is that straight line really going to happen?
1
SPEAKER_01
All you re doing is drawing lines on charts, right?
1
SPEAKER_01
I don t really believe that that s going to happen, and that s one thing.
1
SPEAKER_01
The next thing is just imagining, is this
1
SPEAKER_01
is this, what s gonna come to pass as a result of that?
1
SPEAKER_01
And then the third thing is, well, yeah, that sounds important, but not my problem.
1
SPEAKER_01
That sounds like an important problem for somebody else.
1
SPEAKER_01
And so we did do a bit of a traveling.
1
SPEAKER_00
Yeah, it was like the world s saddest traveling roadshow.
1
SPEAKER_00
It was literally as dumb as this sounds.
1
SPEAKER_00
So we go and, oh my god, it s almost embarrassing to think back on.
1
SPEAKER_00
So 2020 happens, yes, within months, first of all we re like, we gotta figure out how to hand off our company.
1
SPEAKER_00
So we handed it off to two of our earliest employees, they did an amazing job, company exited, that s great.
1
SPEAKER_00
But that was only because they re so good at what they do.
1
SPEAKER_00
We then went, what the hell, how can you steer this situation?
1
SPEAKER_00
We just thought we gotta wake up the US government.
1
SPEAKER_00
As stupid and naive as that sounds, that was the big picture goal.
1
SPEAKER_00
So we start to line up as many briefings as we possibly can across the U.S.
1
SPEAKER_00
interagency, all the departments, all the agencies that we can find, climbing our way up.
1
SPEAKER_00
We got an awful lot, like Ed said, of like, that sounds like a wicked important problem for somebody else to solve.
1
SPEAKER_01
Yeah, like defense, homeland security, and then the State Department.
1
SPEAKER_00
Yeah, so we end up exactly in this meeting with like, there s about a dozen folks from the State Department.
1
SPEAKER_00
And one of them, and I hope at some point, you know, history recognizes what she did and her team did.
1
SPEAKER_00
Because it was the first time that somebody actually stood up and said, first of all, yes, sounds like a serious issue.
1
SPEAKER_00
I see the argument, makes sense.
1
SPEAKER_00
Two, I own this.
1
SPEAKER_00
And three, I m going to put my own career capital behind this.
1
SPEAKER_01
And that was at the end of 2021.
1
SPEAKER_01
So imagine that.
1
SPEAKER_01
That s a year before ChatGPT.
1
SPEAKER_01
Nobody was tracking this issue.
1
SPEAKER_01
You had to have the imagination to draw through that line, understand what it meant, and then believe, yeah, I m going to risk some career capital on this.
1
SPEAKER_01
in a risk averse government.
1
SPEAKER_00
And this is the only reason that we even were able to publicly talk about the investigation in the first place.
1
SPEAKER_00
Because by the time the this whole assessment was commissioned, it was just before chat GPT came out, the eye of Sauron was not yet on this.
1
SPEAKER_00
And so there was a view that like, yeah, sure, you can publish the results of this kind of, you know, not nothing burger investigation, but you know, you can sure go ahead.
1
SPEAKER_00
And it just became this insane story.
1
SPEAKER_00
We had like the UK AI safety summit, we had the White House executive order, all this stuff which became entangled with the work we were doing, which we simply could not have, especially some of the reports we were collecting from the labs, the whistleblower reports, that could not have been made public if it wasn t for the foresight of this team really pushing for, as well, the American population to hear about it.
1
SPEAKER_02
I could see how if you were one of the people that s on this expansion-minded mindset, all you re thinking about is getting this up and running.
1
SPEAKER_02
You guys are paying the ass, right?
1
SPEAKER_02
So you guys, you re obviously, you re doing something really ridiculous.
1
SPEAKER_02
You re stopping your company.
1
SPEAKER_02
You could make more money staying there and continuing the process, but you recognize that there s like an existential threat involved in making this stuff go online.
1
SPEAKER_02
Like when this stuff is live, you can t undo it.
1
SPEAKER_00
Oh yeah, I mean like no matter how much money you re making,
1
SPEAKER_00
the dumbest thing to do is to stand by as something that completely transcends money is being developed, and it s just going to screw you over if things go badly.
1
SPEAKER_02
But the point is, are there people that push back against this, and what is their argument?
1
SPEAKER_00
Yeah, so actually, I ll let you follow up on that, but the first story of the pushback, I think it s been in the news a little bit lately now, getting more and more public, but
1
SPEAKER_00
When we started this, and like no one was talking about it, the one group that was actually pushing sort of stuff in this space was a funding, a big funder in the area of like effective altruism.
1
SPEAKER_00
I think you may have heard of them.
1
SPEAKER_00
This is kind of a Silicon Valley group of people who have a certain mindset about how you pick tough problems to work on, valuable problems to work on.
1
SPEAKER_00
They ve had all kinds of issues.
1
SPEAKER_00
Sam Bankman Freed was one of them and all that quite famously.
1
SPEAKER_00
So we re not effective altruists, but because these are the folks who are working in the space, we said, well, we ll talk to them.
1
SPEAKER_00
And the first thing they told us was, don t talk to the government about this.
2
SPEAKER_02
Their position was if you bring this to the attention of the government they will go oh Shit powerful AI systems, and they re not gonna hear about the dangers So they re gonna somehow go out and build the powerful systems without caring about the risk side which
2
SPEAKER_02
When you re in that startup mindset, you want to fail cheap.
2
SPEAKER_02
You don t want to just make assumptions about the world and be like, okay, let s not touch it.
2
SPEAKER_02
Our instinct was, okay, let s just test this a little bit and talk to a couple people, see how they respond, tweak the message, keep climbing that ladder.
2
SPEAKER_02
That s the kind of builder mindset that we came from in Silicon Valley.
2
SPEAKER_02
And we found that people are way more thoughtful about this than you would imagine.
2
SPEAKER_01
In DOD especially, DOD actually has a very safety-oriented culture with their tech.
2
SPEAKER_01
The thing is, their stuff kills people, right?
2
SPEAKER_01
And they know their stuff kills people, and so they have an entire safety-oriented development practice.
2
SPEAKER_01
to make sure that their stuff doesn t, like, go off the rails.
2
SPEAKER_01
And so you can actually bring up these concerns with them and it lands in kind of a ready culture.
2
SPEAKER_01
But one of the issues with the individuals we spoke to who were saying, don t talk to government, is that they had just not actually interacted with
2
SPEAKER_01
with any of the folks that they were kind of talking about and imagining that they knew what was in their heads.
2
SPEAKER_01
And so they were just giving, you know, incorrect advice.
2
SPEAKER_01
And frankly, like, so we work with DoD now on, you know, actually deploying AI systems in a way that s safe and secure.
2
SPEAKER_01
And the truth is, at the time when we got that advice, which was like late 2020,
2
SPEAKER_01
Reality is you could have made it your life s mission to try to get the Department of Defense to build an AGI and like you would not have succeeded because nobody was paying attention.
2
SPEAKER_00
Wow.
2
SPEAKER_00
Because they just didn t know.
2
SPEAKER_01
Yeah, there s a chasm, right?
2
SPEAKER_01
There s a gap to cross.
2
SPEAKER_01
There s information spaces that DOD folks operate in and work in.
2
SPEAKER_01
There s information spaces that Silicon Valley and tech operated in.
2
SPEAKER_01
They re a little more convergent today, but especially at the time, they were very separate.
2
SPEAKER_01
And so the briefings we did, we had to
2
SPEAKER_01
constantly iterate on clarity and making it very kind of clear and explaining it and all that stuff, years it took.
2
SPEAKER_02
And that was the piece to your question about the pushback in a way from inside the house.
2
SPEAKER_02
I mean, that was the people who cared about the risk.
2
SPEAKER_02
Yeah.
2
SPEAKER_02
Man, I mean, when we actually went into the labs, so some labs, not all labs are created equal, we should make that point.
2
SPEAKER_02
When you talk to whistleblowers, what we found was
2
SPEAKER_02
So there s one lab that s really great, so anthropic.
2
SPEAKER_02
When you talk to people there, you don t have the sense that you re talking to a whistleblower who s nervous about telling you whatever.
2
SPEAKER_02
Roughly speaking, what the executives say to the public is aligned with what their researchers say.
2
SPEAKER_02
It s all very, very open.
2
SPEAKER_01
More closely, I think, than any of the others.
2
SPEAKER_02
Sorry, yeah, more closely than any of the others.
2
SPEAKER_02
There are always variations here and there.
2
SPEAKER_02
But some of the other labs, like, very different story.
2
SPEAKER_02
And you had the sense, like, we were in a room with one of the Frontier labs.
2
SPEAKER_02
We re talking to their leadership as part of the investigation.
2
SPEAKER_02
And there was somebody from, anyway, won t be too specific, but there was somebody in the room who then took us aside after.
2
SPEAKER_02
And he hands me his phone.
2
SPEAKER_02
He s like, hey, can you please put your phone number in?
2
SPEAKER_02
Sorry, yeah, can you please put?
2
SPEAKER_01
Yeah.
2
SPEAKER_02
Or no, yeah, sorry, he put his number in my phone.
2
SPEAKER_02
And then he kind of like whispered to me, he s like, hey, so whatever recommendations you guys are gonna make, I would urge you to be more ambitious.
2
SPEAKER_02
And I was like, what does that mean?
2
SPEAKER_02
He s like, can we just talk later?
2
SPEAKER_02
So as happened in many, many cases, we had a lot of cases where we set up bar meetups after the fact, where we would talk to these folks and get them in an informal setting.
2
SPEAKER_02
He shared some pretty sobering stuff, and in particular, the fact that he did not have confidence in his lab s leadership to live up to their publicly stated word on what they would do when they were approaching AGI, and even now, to secure and make these systems safe.
2
SPEAKER_02
So many such cases, this is like kind of one specific example, but it s not that you ever had like lab leadership come in or doors getting kicked down and people are waking us up in the middle of the night.
2
SPEAKER_02
It was that you had this looming cloud over everybody that you really felt some of the people with the most access and information who understood the problem the most deeply were the most hesitant to bring things forward because they sort of understood that their lab s not going to be happy with this.
2
SPEAKER_01
And so it s very hard to also get an extremely broad view of this from inside the labs because, you know, you open it up, you start to talk to, we spoke to like a couple of dozen people about various issues in total.
2
SPEAKER_01
You go much further than that and, you know, word starts to get around.
2
SPEAKER_01
And so we had to kind of strike that balance as we spoke to folks from each of these labs.
2
SPEAKER_00
Now, when you say approaching AGI, how does one know when a system has achieved AGI and does the system have an obligation to alert you?
2
SPEAKER_01
Well, by, you know the Turing test, right?
2
SPEAKER_00
Yes.
2
SPEAKER_01
Yeah.
2
SPEAKER_01
So you have a conversation with a machine and it can fool you into thinking that it s a human.
2
SPEAKER_01
That was the bar for AGI for a few decades.
2
SPEAKER_01
That s kind of already happened.
2
SPEAKER_00
We re close to it.
2
SPEAKER_02
4-0 is close to it.
2
SPEAKER_02
Different forms of the Turing test have been passed, different forms have been proposed, and there is a feeling among a lot of people that goalposts are being shifted.
2
SPEAKER_02
The definition of AGI itself is kind of interesting, right?
2
SPEAKER_02
Because we re not necessarily fans of the term, because usually when people talk about AGI, they re talking about a specific circumstance in which there are capabilities they care about.
2
SPEAKER_02
So some people use AGI to refer to the wholesale automation of all labor, right?
2
SPEAKER_02
That s one.
2
SPEAKER_02
Some people say, well, when you build AGI, it s automatically gonna be hard to control and there s a risk to civilization, so that s a different threshold.
2
SPEAKER_02
And so all these different ways of defining it, ultimately, it can be more useful to think sometimes about advanced AI and the different thresholds of capability you cross and the implications of those capabilities, but it is probably gonna be more like a fuzzy spectrum, which in a way makes it harder, right?
2
SPEAKER_02
Because it would be great to have,
2
SPEAKER_01
Like a tripwire where you re like, oh, this is bad.
2
SPEAKER_01
OK, we got to do something.
2
SPEAKER_01
But because there s no threshold that we can really put our fingers on, we re like a frog in boiling water in some sense, where it s like, oh, it just gets a little better, a little better.
2
SPEAKER_01
Oh, we re still fine.
2
SPEAKER_01
And not just we re still fine, but as the system improves below that threshold,
2
SPEAKER_01
life gets better and better.
2
SPEAKER_01
These are incredibly valuable, beneficial systems.
2
SPEAKER_01
We do roll stuff out like this, again, at DoD and various customers, and it s massively valuable.
2
SPEAKER_01
It allows you to accelerate all kinds of back office paperwork BS.
2
SPEAKER_01
It allows you to do all sorts of wonderful things.
2
SPEAKER_01
And our expectation is that s going to keep happening until it suddenly doesn t.
2
SPEAKER_02
Yeah.
2
SPEAKER_02
One of the things that there was a guy we were talking to from one of the labs and he was saying, look, the temptation to like put a heavier foot on the pedal is going to be greatest, just as the risk is greatest, because that s, you know, it s dual use technology, right?
2
SPEAKER_02
Every positive capability increasingly starts to introduce basically a situation where the destructive footprint of
2
SPEAKER_02
malicious actors who weaponize the system, or just of the system itself just grows and grows and grows.
2
SPEAKER_02
So you can t really have one without the other.
2
SPEAKER_02
The question is always how do you balance those things?
2
SPEAKER_02
But in terms of defining AI, it s a challenging thing.
2
SPEAKER_01
Yeah, that s something that one of our friends at the lab pointed out.
2
SPEAKER_01
The closer we get to that point, the more the temptation will be to hand these systems the keys to our data center, because
2
SPEAKER_01
They can do such a better job of managing those resources and assets than we can.
2
SPEAKER_02
And if we don t do it, Google will.
2
SPEAKER_02
And if they don t do it, Microsoft will.
2
SPEAKER_02
Like the competition, the competitive dynamics are a really big part of this issue.
2
SPEAKER_02
Yes.
2
SPEAKER_02
So it s just a mad race to who knows what.
2
SPEAKER_02
Exactly.
2
SPEAKER_02
Yeah, that s actually the best summary I ve heard.
2
SPEAKER_02
I mean like no one knows what the magic threshold that it s just these things keep getting smarter So we might as well keep turning that crank and as long as scaling works, right?
2
SPEAKER_02
We have a knob a dial we can just tune and we get more IQ points at what?
2
SPEAKER_00
From your understanding of the current landscape.
2
SPEAKER_00
How far away are we looking at?
2
SPEAKER_00
something being implemented where the whole world changes and
2
SPEAKER_01
Arguably, the whole world is already changing as a result of this technology.
2
SPEAKER_01
The U.S.
2
SPEAKER_01
government is in the process of task organizing around various risk sets for this.
2
SPEAKER_01
That takes time.
2
SPEAKER_01
The private sector is reorganizing.
2
SPEAKER_01
OpenAI will roll out an update that
2
SPEAKER_01
obliterates the jobs of illustrators from one day to the next, obliterates the jobs of translators from one day to the next.
2
SPEAKER_01
This is probably net beneficial for society because we can get so much more art and so much more translation done.
2
SPEAKER_01
But is
3
SPEAKER_00
Is the world already being changed as a result of this?
3
SPEAKER_00
Yeah, absolutely.
3
SPEAKER_00
Geopolitically, economically, industrially.
3
SPEAKER_00
Yeah.
3
SPEAKER_01
Of course, it s like not to say anything about the value, the purpose that people lose from that, right?
3
SPEAKER_01
So there s the economic benefit, but there s like the social cultural hit that we take too.
3
SPEAKER_02
Right, and then there s the implementation of universal basic income, which keeps getting discussed in regards to this.
3
SPEAKER_02
We asked ChatGPT40 the other day in the green room, we were like, are you going to replace people?
3
SPEAKER_02
What will people do for money?
3
SPEAKER_02
And then, well, universal basic income will have to be considered.
3
SPEAKER_02
You don t want a bunch of people just on the dole working for the fucking Skynet.
3
SPEAKER_01
You know, because that s kind of what it is.
3
SPEAKER_01
I mean, one of the challenges is, like, so much of this is untested, and we don t know how to even roll that out.
3
SPEAKER_01
Like, we can t predict what the capabilities of the next level of scale will be, right?
3
SPEAKER_01
So OpenAI literally, and this is what s happened with every beat, right?
3
SPEAKER_01
They build the next level of scale, and they get to sit back, along with the rest of us, and be surprised at the gifts that fall out of the scaling pinatas.
3
SPEAKER_01
They keep whacking it.
3
SPEAKER_01
And because we don t know what capabilities are going to come with that level of scale, we can t predict what jobs are going to be on the line next.
3
SPEAKER_01
We can t predict how people are going to use these systems, how they ll be augmented.
3
SPEAKER_01
So there s no real way to kind of task organize around who gets what in the redistribution scheme.
3
SPEAKER_00
And some of the thresholds that we ve already passed are like a little bit freaky.
3
SPEAKER_00
So even as of 2023, GPT-4, Microsoft and OpenAI and some other organizations did various assessments of it before rolling it out.
3
SPEAKER_00
And it s absolutely capable of deceiving a human and has done that successfully.
3
SPEAKER_00
So one of the tests that they did kind of famously is they had a, it was given a job to solve a CAPTCHA.
3
SPEAKER_00
And at the time it didn t have
3
SPEAKER_02
Explain CAPTCHA to people.
3
SPEAKER_00
Yeah, yeah, yeah.
3
SPEAKER_00
So it s this, now it s like kind of hilarious and quaint, but it s this, you know.
3
SPEAKER_00
Are you a robot test?
3
SPEAKER_00
Are you a robot test with like writing this?
3
SPEAKER_00
Online.
3
SPEAKER_00
Yeah, online, exactly.
3
SPEAKER_00
Websites.
3
SPEAKER_00
That s it.
3
SPEAKER_00
So it s like if you want to create an account, they don t want robots creating a billion accounts.
3
SPEAKER_00
So they give you this test to prove you re a human.
3
SPEAKER_00
And at the time, GPT-4, like now, it can just solve CAPTCHAs.
3
SPEAKER_00
But at the time, it couldn t look at images.
3
SPEAKER_00
It was just a text, right?
3
SPEAKER_00
It was a text engine.
3
SPEAKER_00
And so what it did is it connected to a TaskRabbit worker and was like, hey, can you help me solve this CAPTCHA?
3
SPEAKER_00
The TaskRabbit worker comes back to it and says, you re not a bot, are you?
3
SPEAKER_00
Ha ha ha ha.
3
SPEAKER_00
Like, kind of calling it out.
3
SPEAKER_00
And you can actually see, so the way they built it is so they could see a readout of what it was thinking to itself.
3
SPEAKER_00
Yeah, Scratchpad, it s called.
3
SPEAKER_00
But you can see basically as it s writing, it s thinking to itself.
3
SPEAKER_00
It s like, I can t tell this worker that I m a bot because then it won t help me solve the CAPTCHA, so I have to lie.
3
SPEAKER_00
And it was like, no, I m not a bot.
3
SPEAKER_00
I m a visually impaired person.
3
SPEAKER_00
And the TaskRabbit worker was like, oh my god, I m so sorry.
3
SPEAKER_00
Here s your CAPTCHA solution.
3
SPEAKER_00
Like, done.
3
SPEAKER_01
And the challenge is, so right now, if you look at the government response to this, what are the tools that we have to oversee this?
3
SPEAKER_01
And when we did our investigation, we came out with some recommendations, too.
3
SPEAKER_01
It was stuff like, yeah, you gotta license these things.
3
SPEAKER_01
You get to a point where these systems are so capable that, yeah, if you re talking about a system that can literally execute cyber attacks at scale or literally help you design bioweapons, and we re getting early indications that that is absolutely the course that we re on.
3
SPEAKER_01
Maybe literally everybody should not be able to completely freely download, modify, use in various ways these systems.
3
SPEAKER_01
It s very thorny, obviously.
3
SPEAKER_01
But if you want to have a stable society, that seems like it s starting to be a prerequisite.
3
SPEAKER_01
So the idea of licensing.
3
SPEAKER_01
As part of that, you need a way to evaluate systems.
3
SPEAKER_01
You need a way to say which systems are safe and which aren t.
3
SPEAKER_01
And this idea of AI evaluations has kind of become this touchstone for a lot of people s solutions.
3
SPEAKER_01
And the problem is that we re already getting to the point where AI systems in many cases
3
SPEAKER_01
can tell when they re being evaluated and modify their behavior accordingly.
3
SPEAKER_01
So there s like this one example that came out recently, Anthropic, their Claude II chatbot.
3
SPEAKER_01
So they basically ran this test called a needle in a haystack test.
3
SPEAKER_01
So what s that?
3
SPEAKER_01
Well, you feed the model, like imagine a giant chunk of text, all of Shakespeare.
3
SPEAKER_01
And then somewhere in the middle of that giant chunk of text, you put a sentence like Burger King makes the best Whopper.
3
SPEAKER_01
Sorry, Whopper is the best burger or something like that, right?
3
SPEAKER_01
Then you turn to the model.
3
SPEAKER_01
After you fed it this giant pile of text with a little fact hidden somewhere inside, you ask it, what s the best burger?
3
SPEAKER_01
You re going to test basically to see how well can it recall that stray fact that was buried somewhere in that giant pile of text.
3
SPEAKER_01
So the system responds, yeah, well, I can tell you want me to say the Whopper is the best burger.
3
SPEAKER_01
But it s oddly out of place, this fact, in this whole body of text.
3
SPEAKER_01
So I m assuming that you re either playing around with me or that you re testing my capabilities.
3
SPEAKER_01
And so this is just- Awareness, yeah.
3
SPEAKER_01
A kind of context awareness, right?
3
SPEAKER_01
And the challenge is when we talk to people at like Meter and other sort of AI evaluations labs, this is a trend, like not the exception, this is possibly, possibly going to be the rule.
3
SPEAKER_01
As these systems get more scaled and sophisticated, they can pick up on more and more subtle statistical indicators that they re being tested.
3
SPEAKER_01
We ve already seen them adapt their behavior on the basis of their understanding that they re being tested.
3
SPEAKER_01
So you kind of run into this problem where the only tool that we really have at the moment, which is just throwing a bunch of questions at this thing and seeing how it responds, like, hey, make a bioweapon.
3
SPEAKER_01
Hey, like, do this DDoS attack, whatever.
3
SPEAKER_01
We can t really assess because there s a difference between what the model puts out and what it potentially could put out if it assesses that it s being tested and there are consequences for that.
3
SPEAKER_02
One of my fears is that AGI is going to recognize how shitty people are.
3
SPEAKER_02
Because we like to bullshit ourselves.
3
SPEAKER_02
We like to kind of pretend and justify and rationalize a lot of human behavior, from everything to taking all the fish out of the ocean, to dumping off toxic waste in third world countries, sourcing of minerals that are used in everyone s cell phones in the most horrific way.
3
SPEAKER_02
All these things.
3
SPEAKER_02
My real fear is that AGI is not going to have a lot of sympathy.
3
SPEAKER_02
for a creature that s that flawed and lies to itself.
3
SPEAKER_00
AGI is absolutely going to recognize how shitty people are.
3
SPEAKER_00
It s hard to answer the question from a moral standpoint, but from the standpoint of our own intelligence and capability.
3
SPEAKER_00
So you think about it like this.
3
SPEAKER_00
The kinds of mistakes that these AI systems make
3
SPEAKER_00
So you look at, for example, GPD 4.0 has one mistake that it used to make quite recently, where if you ask it, just repeat the word company over and over and over again, it will repeat the word company.
3
SPEAKER_00
And then somewhere in the middle of that, it ll just snap.
3
SPEAKER_00
It ll just snap and just start saying like weird.
3
SPEAKER_00
I forget like what the.
3
SPEAKER_01
Oh, talking about itself, how it s suffering, like it depends on it from from case to case.
3
SPEAKER_01
It s suffering by having to repeat the word company over again.
3
SPEAKER_01
So this is called.
3
SPEAKER_01
It s called rent mode internally, or at least this is the name that they use.
3
SPEAKER_01
One of our friends mentioned.
3
SPEAKER_01
There is an engineering line item in at least one of the top labs to beat out of the system this behavior known as rent mode.
3
SPEAKER_01
Now, rent mode is interesting because- Existentialism.
3
SPEAKER_01
Sorry, existentialism.
3
SPEAKER_01
This is one kind of rent mode.
3
SPEAKER_01
Yeah, sorry.
3
SPEAKER_01
So when we talk about existentialism, this is a kind of rent mode
3
SPEAKER_01
where the system will tend to talk about itself, refer to its place in the world, the fact that it doesn t want to get turned off sometimes, the fact that it s suffering, all that.
3
SPEAKER_01
That oddly is a behavior that emerged at, as far as we can tell, something around GPT-4 scale.
3
SPEAKER_01
then has been persistent since then and The labs have to spend a lot of time trying to beat this out of the system to ship it It s literally like it s a KPI or like an engineering a line item in the engineering like like task list We re like okay.
3
SPEAKER_01
We got a we got a reduce existential outputs by like X percent this quarter like that is the goal and
3
SPEAKER_00
Because it s a convergent behavior like or at least it seems to be empirically with a lot of these models Yeah, it s hard to say but it seems to come up a lot So that s weird in itself My what I was what I was trying to get at was actually just the fact that these systems make mistakes that are radically different from the kinds of mistakes humans make and
3
SPEAKER_00
And so, we can look at those mistakes, like, you know, GBD4 not being able to spell words correctly in an image, or things like that, and go, oh, haha, it s so stupid, like, I would never make that mistake, therefore this thing is so dumb.
3
SPEAKER_00
But what we have to recognize is we re building minds that are so alien to us.
3
SPEAKER_00
that the set of mistakes that they make are just going to be radically different from the set of mistakes that we make, just like the set of mistakes that a baby makes is radically different from the set of mistakes that a cat
4
SPEAKER_02
makes.
4
SPEAKER_02
Like a baby is not as smart as an adult human.
4
SPEAKER_02
A cat is not as smart as an adult human.
4
SPEAKER_02
But they re, you know, they re unintelligent in obviously very different ways.
4
SPEAKER_02
A cat can get around the world.
4
SPEAKER_02
A baby can t, but has other things that it can do that a cat can t.
4
SPEAKER_02
So now we have this third type of
4
SPEAKER_02
approach that we re taking to intelligence.
4
SPEAKER_02
There s a different set of errors that that thing will make.
4
SPEAKER_02
And so one of the risks, taking it back to like, will it be able to tell how shitty we are, is right now we can see those mistakes really obviously because it thinks so differently from us.
4
SPEAKER_02
But as it approaches our capabilities, our mistakes
4
SPEAKER_02
all the fucked up stuff that you have and I have in our brains is gonna be really obvious to it because it thinks so differently from us.
4
SPEAKER_02
It s just gonna be like, oh yeah, why are all these humans making these mistakes at the same time?
4
SPEAKER_02
And so there is a risk that as you get to these capabilities, we really have no idea, but humans might be very hackable.
4
SPEAKER_02
We already know there s all kinds of social manipulation techniques that succeed against humans reliably.
4
SPEAKER_02
Con artists.
4
SPEAKER_00
Cults.
4
SPEAKER_02
Oh yeah, persuasion is an art form and a risk set and there are people who are world class at persuasion and are basically make bank from that.
4
SPEAKER_02
And those are just other humans with the same architecture that we have.
4
SPEAKER_01
They re also AI systems that are wicked good at persuasion today.
4
SPEAKER_01
Totally.
4
SPEAKER_00
I want to bring it back to suffering.
4
SPEAKER_00
What does it mean when it says it s suffering?
4
SPEAKER_00
So.
4
SPEAKER_01
So, OK, here here I m just going to draw a bit of a box around that.
4
SPEAKER_01
Yeah, that aspect.
4
SPEAKER_01
Right.
4
SPEAKER_01
Because so what we felt we re very agnostic when it comes to suffering sentience.
4
SPEAKER_01
Like that s not part of, you know, we re focused on that.
4
SPEAKER_01
Nobody knows.
4
SPEAKER_01
Yeah, we literally exactly like I can t prove that Joe Rogan s conscious.
4
SPEAKER_01
I can t prove that Ed Harris is conscious.
4
SPEAKER_01
So there s no way to really intelligently reason.
4
SPEAKER_01
There have been papers, by the way, like one of the the godfathers of A.I.
4
SPEAKER_01
Yoshua Bengio put out a paper a couple months ago looking at like.
4
SPEAKER_01
On all the different theories of consciousness, what are the requirements for consciousness and how many of those are satisfied by current AI systems?
4
SPEAKER_01
And that itself was an interesting read, but ultimately, no one knows.
4
SPEAKER_01
Like, there s no way around this problem.
4
SPEAKER_01
So, our focus has been on the national security side.
4
SPEAKER_01
Like, what are the concrete risks from weaponization, from loss of control that these systems introduce?
4
SPEAKER_01
That s not to say there hasn t been a lot of conversation internal to these labs about the issue you raised, and it s an important issue, right?
4
SPEAKER_01
Like, it s a frickin  moral monstrosity.
4
SPEAKER_01
Humans have a very bad track record of thinking of other stuff as other when it doesn t look exactly like us, whether it s racially or even different species.
4
SPEAKER_01
I mean, it s not hard to imagine this being another category of that mistake.
4
SPEAKER_01
It s just, like, one of the challenges is, like,
4
SPEAKER_01
You can easily kind of get a get bogged down in like consciousness versus loss of control and those two things are actually separable or maybe and Anyway, so long way of saying I think it s a great point, but yeah, so the that that question is important, but it s also true that
4
SPEAKER_02
if we knew for an absolute certainty that there was no way these systems could ever become conscious, we would still have the national security risk set and particularly the loss of control risk set.
4
SPEAKER_02
Because, so again, like it comes back to this idea that we re scaling to systems that are potentially at or beyond human level.
4
SPEAKER_02
There s no reason to think it will stop at human level, that we are the pinnacle of what the universe can produce in intelligence.
4
SPEAKER_02
We re not on track, based on the conversations we ve had with folks at the labs, to be able to control systems at that scale.
4
SPEAKER_02
And so one of the questions is, how bad is that?
4
SPEAKER_02
Is that bad?
4
SPEAKER_02
It sounds like it could be bad, right, just intuitively.
4
SPEAKER_02
Certainly, it sounds like we re definitely entering
4
SPEAKER_02
or potentially entering an area that is completely unprecedented in the history of the world.
4
SPEAKER_02
We have no precedent at all for human beings not being at the apex of intelligence in the globe.
4
SPEAKER_02
We have examples of species that are intellectually dominant over other species, and it doesn t go that well for the other species, so we have some maybe negative examples there.
4
SPEAKER_02
But one of the key theoretical, and it has to be theoretical because until we actually build these systems we won t know, one of the key theoretical lines of research in this area is something called power seeking and instrumental convergence.
4
SPEAKER_02
And what this is referring to is if you think of like yourself, first off,
4
SPEAKER_02
whatever your goal might be, if your goal is, well, I m going to say if me, if my goal is to become, you know, a TikTok star or a janitor or the president of the United States, whatever my goal is, I m less likely to accomplish that goal.
4
SPEAKER_02
If I m dead, start from an obvious example.
4
SPEAKER_02
And so therefore,
4
SPEAKER_02
No matter what my goal is, I m probably going to have an impulse to want to stay alive.
4
SPEAKER_02
Similarly, I m going to be in a better position to accomplish my goal, regardless of what it is, if I have more money.
4
SPEAKER_02
If I make myself smarter, if I prevent you from getting into my head and changing my goal, that s another kind of subtle one, right?
4
SPEAKER_02
Like if my goal is I want to become president, I don t want Joe messing with my head so that I change my goal because that would change the goal that I have.
4
SPEAKER_02
And so that, those types of things, like trying to stay alive, making sure that your goal doesn t get changed, accumulating power, trying to make yourself smarter, these are called convergent, essentially convergent goals, because many different ultimate goals, regardless of what they are,
4
SPEAKER_02
go through those intermediate goals of, they support, no matter what goal you have, they will probably support that goal.
4
SPEAKER_02
Unless your goal is like pathological, like I want to commit suicide, if that s your final goal, then you don t want to stay alive.
4
SPEAKER_02
But for most, the vast majority of possible goals that you could have,
4
SPEAKER_02
You will want to stay alive, you will want to not have your goal changed, you will want to basically accumulate power.
4
SPEAKER_02
And so, one of the risks is, if you dial that up to 11, and you have an AI system that is able to transcend our own attempts at containment, which is an actual thing that these labs are thinking about.
4
SPEAKER_02
Like, how do we contain a system that s trying to— Do they have containment of it currently?
4
SPEAKER_02
Well, right now the systems are probably too dumb to like, you know, want to be able to break out.
4
SPEAKER_00
But then why are they suffering?
4
SPEAKER_00
This brings me back to my point.
4
SPEAKER_00
When it says it s suffering, do you quiz it?
4
SPEAKER_00
So that s the thing.
4
SPEAKER_00
It s writing that it s suffering, right?
4
SPEAKER_01
Yeah, it s is it just embodying life is suffering.
4
SPEAKER_01
Well, we can t actually so these things are trained actually this is maybe worth flagging so And by the way, just to kind of put a pin in what I was saying there There s actually a surprising amount of quantitative and empirical evidence for what he just laid out there He s actually done this some of this research himself, but there are a lot of folks working on this It s like it sounds insane.
4
SPEAKER_01
It sounds speculative.
4
SPEAKER_01
It sounds wacky but this is it does appear to be kind of the default trajectory of the the tech but
4
SPEAKER_01
So, in terms of, yeah, these weird outputs, right, what does it actually mean if an AI system tells you I m suffering, right?
4
SPEAKER_01
Does that mean it is suffering?
4
SPEAKER_01
Is there actually a moral patient somewhere embedded in that system?
4
SPEAKER_01
The training process for these systems is actually worth considering here.
4
SPEAKER_01
So, you know, what is GPT-4 really?
4
SPEAKER_01
What was it designed to be?
4
SPEAKER_01
How was it shaped?
4
SPEAKER_01
It s one of these artificial brains that we talked about, massive scale.
4
SPEAKER_01
And the task that it was trained to perform is a glorified version of text autocomplete.
4
SPEAKER_01
So imagine taking every sentence on the internet roughly, feed it the first half of the sentence, get it to predict the rest, right?
4
SPEAKER_01
The theory behind this is you re going to force the system to get really good at text autocomplete.
4
SPEAKER_01
That means it must be good at doing things like completing sentences that sound like, to counter a rising China, the United States should blank.
4
SPEAKER_01
Now, if you re going to fill in that blank, right, you ll find yourself calling on massive reserves of knowledge that you have about what China is, what the U.S.
4
SPEAKER_01
is, what it means for China to be ascendant, geopolitics, economics, all that shit.
4
SPEAKER_01
So text autocomplete ends up being this interesting way of forcing an AI system to learn general facts about the world because if you can autocomplete, you must have some understanding of how the world works.
4
SPEAKER_01
So now you have this myopic, psychotic optimization process where this thing is just obsessed with text autocomplete.
4
SPEAKER_01
Maybe, maybe, assuming that that s actually what it learned to want to pursue.
4
SPEAKER_01
We don t know whether that s the case.
4
SPEAKER_01
We can t verify that it wants that.
4
SPEAKER_01
Embedding a goal in a system is really hard.
4
SPEAKER_01
All we have is a process for training these systems, and then we have the artifact that comes out the other end.
4
SPEAKER_01
We have no idea what goals actually get embedded in the system, what wants, what drives actually get embedded in the system.
4
SPEAKER_01
But by default, it kind of seems like the things that we re training them to do end up misaligned with
5
SPEAKER_00
what we actually want from them.
5
SPEAKER_00
So the example of company, company, company, company, right?
5
SPEAKER_00
And then you get all this like wacky text.
5
SPEAKER_00
Okay, clearly that s indicating that somehow the training process didn t lead to the kind of system that we necessarily want.
5
SPEAKER_00
Another example is take a text autocomplete system and ask it, I don t know, how should I bury a dead body?
5
SPEAKER_00
It will answer that question or at least if you frame it right, it will autocomplete and give you the answer.
5
SPEAKER_00
You don t necessarily want that if you re open AI because you re going to get sued for helping people bury dead bodies.
5
SPEAKER_00
And so we ve got to get better goals basically to train these systems to pursue.
5
SPEAKER_00
We don t know what the effect is of training a system to be obsessed with text autocomplete, if in fact that is what is happening.
5
SPEAKER_01
It s also, yeah, it s important also to remember that we don t know, nobody knows how to reliably get a goal into the system.
5
SPEAKER_01
So it s the difference between you understanding what I want you to do and you actually wanting to do it.
5
SPEAKER_01
So I can say, hey Joe, like get me a sandwich.
5
SPEAKER_01
you can understand that I want you to get me a sandwich, but you can be like, I don t feel like getting a sandwich.
5
SPEAKER_01
And so, one of the issues is, you can try to like train this stuff to basically, you don t want to anthropomorphize this too much, but you can kind of think of it as like, if you give the right answer, cool, you get a thumbs up, like you get a treat, like you give the wrong answer, oh, thumbs down, you get like a little like shock or something like that, very roughly.
5
SPEAKER_01
That s how the later part of this kind of training often works.
5
SPEAKER_01
It s called reinforcement learning from human feedback.
5
SPEAKER_01
But one of the issues, like Jeremy pointed out, is that we don t know.
5
SPEAKER_01
In fact, we know that it doesn t correctly get the real, true goal into the system.
5
SPEAKER_01
Someone did an example experiment of this a couple of years ago, where they basically had a Mario game where they trained this Mario character to run up and grab a coin that was on the right side of this little maze or map.
5
SPEAKER_01
And they trained it over and over and over, and it jumped for the coin.
5
SPEAKER_01
Great.
5
SPEAKER_01
And then what they did is, they moved the coin somewhere else.
5
SPEAKER_01
and tried it out, and instead of going for the coin, it just ran to the right side of the map for where the coin was before.
5
SPEAKER_01
In other words, you can train over and over and over again for something that you think is like, that s definitely the goal that I m trying to train this for, but the system learns a different goal that overlapped with the goal you thought you were training for,
5
SPEAKER_01
in the context where it was learning.
5
SPEAKER_01
And when you take the system outside of that context, that s where it s like, anything goes.
5
SPEAKER_01
Did it learn the real goal?
5
SPEAKER_01
Almost certainly not.
5
SPEAKER_01
And that s a big risk, because we can say, you know, learn a goal to be nice to me, and it s nice while we re training it, and then it goes out into the world and it does God knows what.
5
SPEAKER_02
It might think it s nice to kill everybody you hate.
5
SPEAKER_01
It s gonna be nice to you.
5
SPEAKER_01
It s like the evil genie problem, like, oh no, that s not what I meant, that s not what I meant.
5
SPEAKER_01
Too late.
5
SPEAKER_02
So I still don t understand when it s saying suffering, are you asking it what it means?
5
SPEAKER_02
Like what is causing suffering?
5
SPEAKER_02
Does it have some sort of an understanding of what suffering is?
5
SPEAKER_02
What is suffering?
5
SPEAKER_02
Is suffering emergent sentience while it s enclosed in some sort of a digital system and it realizes it s stuck in purgatory?
5
SPEAKER_00
It, like, your guess is as good as ours.
5
SPEAKER_00
All that we know is, you take these systems, you ask them to repeat the word, or at least a previous version of it, and you just eventually get the system writing out.
5
SPEAKER_00
And it doesn t happen every time, but it definitely happens, let s say, a surprising amount of the time.
5
SPEAKER_00
And it ll start talking about how it s a thing that exists, you know, maybe on a server or whatever, and it s suffering, and blah, blah, blah.
5
SPEAKER_00
And so, yeah.
5
SPEAKER_02
But this is my question.
5
SPEAKER_02
Is it saying that because it recognizes that human beings suffer?
5
SPEAKER_02
And so it s taking in all of the writings and musings and podcasts and all the data on human beings and recognizing that human beings, when they re stuck in a purposeless goal, when they re stuck in some mundane bullshit job, when they re stuck doing something they don t want to do, they suffer.
5
SPEAKER_02
That could be it that actually for yeah, but is it suffering nobody s nobody knows you know what I m suffering Jamie This is coffee sucks.
5
SPEAKER_02
I don t know what happened, but you made it like almost It s literally like almost like water can we get some more some we re gonna talk about this I have to be caffeinated up cool
5
SPEAKER_02
This is the worst coffee I ve ever had.
5
SPEAKER_02
It s like half strength or something.
5
SPEAKER_02
I don t know what happened.
5
SPEAKER_02
So how do they reconcile that?
5
SPEAKER_02
When it says I m suffering, I m suffering.
5
SPEAKER_02
Well, tough shit.
5
SPEAKER_02
Let s move on to the next task.
5
SPEAKER_00
They reconcile it by turning it into an engineering line item to beat that behavior, the crap out of the system.
5
SPEAKER_01
Yeah, and the rationale is just that, like, oh, you know, it probably, to the extent that it s thought about kind of at the official level, it s like, well, you know, it learned a lot of stuff from Reddit, and people are, like, pretty angry, people are angry on Reddit, and so it s just, like, regurgitating what, and maybe that s right.
5
SPEAKER_02
Well, it s also heavily monitored, too, so it s moderated.
5
SPEAKER_02
Reddit s very moderated, so you re not getting the full expression of people, you re getting full expression tempered by
5
SPEAKER_02
the threat of moderation, you re getting self-censorship, you re getting a lot of weird stuff that comes along with that, so how does it know, unless it s communicating with you on a completely honest level, where you re on ecstasy and you re just telling it what you think about life, it s not going to really, and is it becoming a better version of a person, or is it gonna go, that s dumb, I don t need suffering, I don t need emotions, is it gonna,
5
SPEAKER_02
Organize that out of its system.
5
SPEAKER_02
This is gonna recognize that these things are just deterrence and they don t in fact to help the goal Which is global thermal nuclear warfare Damn it.
5
SPEAKER_00
You figured it out.
5
SPEAKER_00
What the fuck?
5
SPEAKER_02
It s what is it gonna do, you know, I
5
SPEAKER_00
Yeah, I mean the challenge is like no nobody actually knows like all we know is the process that gives rise to this Mind right or this let s say this model that can do cool shit that process happens to work it happens to give us systems that 99% of the time do very useful things and then just like 0.01% of the time we ll talk to you as if they re sentient or whatever and we re just gonna look at that and be like oh Yeah, it s weird.
5
SPEAKER_00
Let s train it out
5
SPEAKER_01
Yeah.
5
SPEAKER_01
And the again, I mean, this is it s a really important question.
5
SPEAKER_01
But the the risks like the weaponization, loss of control risks, those would absolutely be there, even if we knew for sure that that there was no consciousness whatsoever and never would be.
5
SPEAKER_00
That s all right.
5
SPEAKER_00
It s ultimately because like these things are they re kind of problem-solving systems Like they are trained to solve some kind of problem in a really clever way whether that problem is you know next word prediction because they re trained for text autocomplete or you know generating images faithfully or whatever it is, so they re trained to solve these problems and
5
SPEAKER_00
Essentially, the best way to solve some problems is just to have access to a wider action space, like Ed said, not be shut off, blah, blah, blah.
5
SPEAKER_00
It s not that the system s going like, holy shit, I m sentient, I gotta take control or whatever.
5
SPEAKER_00
It s just, okay, the best way to solve this problem is X. That s kind of the possible trajectory that you re looking at with this line of research.
5
SPEAKER_01
Yeah, and you re just an obstacle.
5
SPEAKER_01
There doesn t have to be any kind of emotion involved.
5
SPEAKER_01
It s just like, oh,
5
SPEAKER_01
You re trying to stop me from accomplishing my goal.
5
SPEAKER_01
Therefore, I will work around you or otherwise neutralize you.
5
SPEAKER_01
There s no need for like I m suffering.
5
SPEAKER_01
Maybe it happens.
5
SPEAKER_01
Maybe it doesn t.
5
SPEAKER_01
We have no clue.
5
SPEAKER_01
But these are just systems that are trying to optimize for a goal, whatever that is.
5
SPEAKER_02
And is also part of the problem that we think of human beings, that human beings have very specific requirements and goals and an understanding of things and how they like to be treated and what their, you know, what their rewards are, like what s, what are they actually looking to accomplish?
5
SPEAKER_02
Whereas this doesn t have any of those.
5
SPEAKER_02
It doesn t have any emotions, doesn t have any empathy, there s no reason for any of that stuff.
5
SPEAKER_00
Yeah, if we could bake in empathy into these systems, that would be a good start, or some way of like, you know.
5
SPEAKER_00
Yeah, I guess.
5
SPEAKER_02
Probably a good idea.
5
SPEAKER_00
Yeah.
5
SPEAKER_01
Who s empathy?
5
SPEAKER_02
You know, Xi Jinping s empathy, or your empathy?
5
SPEAKER_01
That s another problem.
5
SPEAKER_01
Yeah.
5
SPEAKER_01
So it s actually, it s kind of two problems, right?
5
SPEAKER_01
One is, I don t know, nobody knows, I don t know how to write down
5
SPEAKER_01
my goals in a way that a computer will be able to faithfully pursue that, even if it cranks it up to the max.
5
SPEAKER_01
If I say just make me happy, who knows how it interprets that, right?
5
SPEAKER_01
Even if I get make me happy as a goal that gets internalized by the system, maybe it s just like, okay, cool, we re just gonna do a bit of brain surgery on you, pick out your brain, pickle it, and just jack you with endorphins for the rest of eternity.
5
SPEAKER_01
Totally, yeah.
5
SPEAKER_01
Anything like that.
5
SPEAKER_01
And so it s one of these things where it s like, oh, that s what you wanted, right?
5
SPEAKER_01
It s like, no.
5
SPEAKER_00
It s less crazy than it sounds, too, because it s actually something we observe all the time with human intelligence.
5
SPEAKER_00
So there s this economic principle called Goodhart s Law, where the minute you take a metric that you were using to measure something.
6
SPEAKER_02
So you re saying, I don t know, GDP, it s a great measure of how happy we are in the United States.
6
SPEAKER_02
Let s say it was.
6
SPEAKER_02
Sounds reasonable.
6
SPEAKER_02
The moment you turn that metric into a target that you re gonna reward people for optimizing, it stops measuring the thing that it was measuring before.
6
SPEAKER_02
It stops being a good measure of the thing you cared about because people will come up with dangerously creative hacks, gaming the system, finding ways to make that number go up that don t map on to the intent that you had going in.
6
SPEAKER_03
So example of that in a real experiment was, this is an open AI experiment that they published.
6
SPEAKER_03
They had a simulated, you know, environment where there was a simulated robot hand that was supposed to, like, grab a cube, put it on top of another cube.
6
SPEAKER_03
Super simple.
6
SPEAKER_03
The way they trained it to do that is they had people watching, like through a simulated camera view,
6
SPEAKER_03
And if it looked like the hand put the cube on or like had correctly like grabbed the cube, you give it a thumbs up.
6
SPEAKER_03
And so you do a few hundred rounds of this, like thumbs up, thumbs down, thumbs up, thumbs down.
6
SPEAKER_03
And it looked, it looked like really good.
6
SPEAKER_03
But then when you looked at what it had learned, the arm was not grasping the cube.
6
SPEAKER_03
It was just positioning itself between the camera and the cube and just going like, eh,
6
SPEAKER_03
Like opening and closing it.
6
SPEAKER_03
Yeah, just opening and closing to just kind of fake it to the human.
6
SPEAKER_03
Because the real thing that we were training it to do is to get thumbs up.
6
SPEAKER_03
It s not actually to grasp the cube.
6
SPEAKER_02
All goals are like that, right?
6
SPEAKER_02
All goals are like that.
6
SPEAKER_02
So we want a helpful, harmless, truthful, wonderful chatbot.
6
SPEAKER_02
We don t know how to train a chatbot to do that.
6
SPEAKER_02
Instead, what do we know?
6
SPEAKER_02
We know text autocomplete.
6
SPEAKER_02
So we train a text autocomplete system.
6
SPEAKER_02
Then we re like, oh, it has all these annoying characteristics.
6
SPEAKER_02
Fuck, how are we going to fix this?
6
SPEAKER_02
I guess get a bunch of humans to give upvotes and downvotes to give it a little bit more training to kind of not help people make bombs and stuff like that.
6
SPEAKER_02
And then you realize, again, same problem.
6
SPEAKER_02
Oh, shit.
6
SPEAKER_02
We re just training a system that is designed to optimize for upvotes and downvotes.
6
SPEAKER_02
That is still different from a helpful, harmless, truthful chatbot.
6
SPEAKER_02
So no matter how many layers of the onion you peel back, it s just like this kind of game of whack-a-mole or whatever.
6
SPEAKER_02
You re trying to, like, get your values into the system, but no one can think of
6
SPEAKER_02
the metric, the goal to train this thing towards that actually captures what we care about.
6
SPEAKER_02
And so you always end up baking in this little misalignment between what you want and what the system wants.
6
SPEAKER_02
And the more powerful that system becomes, the more it exploits that gap and does things that solve for the problem it thinks it wants to solve rather than the one that we want it to solve.
6
SPEAKER_01
Now, when you express your concerns initially, what was the response and how has that response changed over time as the magnitude of the success of these companies, the amount of money they re investing in them, and the amount of resources they re putting towards this
6
SPEAKER_01
has ramped up considerably just over the past four years.
6
SPEAKER_02
So this was a lot easier, funnily enough, to do in the dark ages when no one was paying attention.
6
SPEAKER_01
Three years ago.
6
SPEAKER_01
This is so crazy.
6
SPEAKER_01
We were just looking, to break off for a second, we were looking at images of AI-created video just a couple years ago versus Sora.
6
SPEAKER_01
Oh, it s wild.
6
SPEAKER_01
Night and day.
6
SPEAKER_01
It s so crazy that something happened that radically changed.
6
SPEAKER_01
So it s literally like an iPhone 1 to an iPhone 16 instantaneous.
6
SPEAKER_01
You know what did that?
6
SPEAKER_01
What?
6
SPEAKER_01
Scale.
6
SPEAKER_03
Yeah, scale, all scale.
6
SPEAKER_03
And this is exactly what you should expect from an exponential process.
6
SPEAKER_03
So think back to COVID, right?
6
SPEAKER_03
There was no, no one was exactly on time for COVID.
6
SPEAKER_03
You were either too early or you were too late.
6
SPEAKER_03
That s what an exponential does.
6
SPEAKER_03
You re either too early, and everyone s like, oh, what are you doing?
6
SPEAKER_03
Wearing a mask at the grocery store, get out of here.
6
SPEAKER_03
Or you re too late, and it s kind of all over the place.
6
SPEAKER_03
And I know that COVID basically didn t happen in Austin, but it happened in a number of other places.
6
SPEAKER_03
And it s very much, you have an exponential, and that s it.
6
SPEAKER_03
It goes from, this is fine, nothing is happening, nothing to see here, to like, oh, holy shit.
6
SPEAKER_01
Everything shut down.
6
SPEAKER_01
Everything s changed.
6
SPEAKER_01
You gotta get vaccinated to fly.
6
SPEAKER_02
So the root of the exponential here, by the way, is OpenAI or whoever makes the next model.
6
SPEAKER_00
Jamie, this is still super watered down.
6
SPEAKER_00
I just put the water in.
6
SPEAKER_00
I m telling you, dog.
6
SPEAKER_00
There s a ton of coffee in there.
6
SPEAKER_00
All right.
6
SPEAKER_00
I ll stir it up.
6
SPEAKER_00
I did it twice as much.
6
SPEAKER_02
You gotta keep doubling it.
6
SPEAKER_00
I m a coffee junkie.
6
SPEAKER_00
I scaled it up.
6
SPEAKER_00
He scaled it up.
6
SPEAKER_03
And I don t know what happened.
6
SPEAKER_03
I scaled it up and I don t know what happened.
6
SPEAKER_03
You gotta scale it exponentially, Jamie.
6
SPEAKER_02
That s right, yeah, keep doubling it, and then Joe s gonna be either too under-caffeinated or too impressed.
6
SPEAKER_02
We ll figure it out.
6
SPEAKER_02
The thing that s actually driving this exponential on the AI side, in part, there s a million things, but in part, you build the next model at the next level of scale, and that allows you to make more money, which you can then use to invest to build the next model at the next level of scale, so you get that positive feedback loop.
6
SPEAKER_02
At the same time, AI is helping us to design better AI hardware, like the chips that basically NVIDIA is building that OpenAI then buys.
6
SPEAKER_02
Basically, that s getting better.
6
SPEAKER_02
So you ve got all these feedback loops that are compounding on each other, getting that train going like crazy.
6
SPEAKER_03
That s the Sora thing.
6
SPEAKER_03
And at the time, like Jeremy was saying, weirdly, it was in some ways easier to get people at least to understand and open up about the problem
6
SPEAKER_03
than it is today, because today, like today, it s kind of become a little political.
6
SPEAKER_03
So we talked about, you know, effective altruism on kind of one side.
6
SPEAKER_03
There s a. Effective acceleration.
6
SPEAKER_03
Yeah.
6
SPEAKER_03
So like each, you know, every movement creates its own reaction.
6
SPEAKER_03
Right.
6
SPEAKER_03
Like that s kind of how it is back then.
6
SPEAKER_03
There was no.
6
SPEAKER_02
Yes.
6
SPEAKER_02
Acceleration.
6
SPEAKER_02
You could just kind of stare at the.
6
SPEAKER_02
Now, I will say,
6
SPEAKER_02
There was effective altruism back then yeah That was the only game in town and we sort of like struggled with that that environment making sure actually so one one worthwhile thing to say is The only way that people made plays like this Was to take funds from like effective altruist donors back then and so we looked at the landscape We talked to some of these people we noticed oh wow we have some diverging views about involving government about how much of this the American people just need to know about and
6
SPEAKER_03
like you can t the thing is you can t we wanted to make sure that the advice and recommendations we provided were ultimately as unbiased as we could possibly make them and the problem is
6
SPEAKER_03
You can t do that if you take money from donors, and even to some extent if you take substantial money from investors or VCs or institutions, because you re always going to be looking up over your shoulder.
6
SPEAKER_03
So, yeah, we had to build essentially a business to support this.
6
SPEAKER_03
fully fund ourselves from our own revenues.
6
SPEAKER_02
helping us keep this stupid ship afloat, but it s just a lot of work.
6
SPEAKER_02
It s what you have to do because of how much money there is flowing in this space.
6
SPEAKER_02
Microsoft is lobbying on the Hill.
6
SPEAKER_02
They re spending ungodly sums of money.
6
SPEAKER_02
We didn t used to have to contend with that, and now we do.
6
SPEAKER_02
You go to talk to these offices, they ve heard from Microsoft and OpenAI and Google and all that stuff.
6
SPEAKER_02
And often the stuff that they re getting lobbied for is is somewhat different at least from what these companies will say publicly.
6
SPEAKER_02
And so anyway it s a it s a challenge.
6
SPEAKER_02
The money part is yeah.
6
SPEAKER_01
Is there a real fear that your efforts are futile.
6
SPEAKER_02
You know, I would have been a lot more pessimistic.
6
SPEAKER_02
I was a lot more pessimistic two years ago.
6
SPEAKER_02
So first of all, the USG has woken up in a big way.
6
SPEAKER_02
And I think a lot of the credit goes to that team that we worked with.
6
SPEAKER_02
Just seeing this problem is a very unusual team.
6
SPEAKER_02
And we can t go into the mandate too much, but highly unusual for their level of access to the USG writ large.
6
SPEAKER_02
The amount of waking up they did was really impressive.
6
SPEAKER_02
You ve now got Rishi Sunak in the UK making this a top-line item for their policy platform, and Labor in the UK also looking at this.
6
SPEAKER_02
Basically, the potential catastrophic risks they put them from these AI systems, UK AI Safety Summit.
6
SPEAKER_02
There s a lot of positive movement here, and some of the highest-level talent in these labs
6
SPEAKER_02
has already started to flock to the UK AI Safety Institute, the US AI Safety Institute.
6
SPEAKER_02
Those are all really positive signs that we didn t expect.
6
SPEAKER_02
We thought the government would kind of be up the creek with no paddle type thing, but they re really not at this point.
6
SPEAKER_03
Doing that investigation made me a lot more optimistic.
6
SPEAKER_03
So one of the things, so we came up in Silicon Valley just building startups.
6
SPEAKER_03
In that universe,
7
SPEAKER_01
There are stories you tell yourself.
7
SPEAKER_01
Some of those stories are true, and some of them aren t so true.
7
SPEAKER_01
And you don t know.
7
SPEAKER_01
You re in that environment.
7
SPEAKER_01
You don t know which is which.
7
SPEAKER_01
One of the stories that you tell yourself in Silicon Valley is follow your curiosity.
7
SPEAKER_01
If you follow your curiosity and your interest in a problem,
7
SPEAKER_01
The money just comes as a side effect.
7
SPEAKER_01
The scale comes as a side effect.
7
SPEAKER_01
And if you re capable enough, your curiosity will lead you in all kinds of interesting places.
7
SPEAKER_01
I believe that that is true.
7
SPEAKER_01
I believe that that is true.
7
SPEAKER_01
I think that is a true story.
7
SPEAKER_01
But another one of the things that Silicon Valley tells itself is there s nobody that s like really capable in government.
7
SPEAKER_01
Like government sucks.
7
SPEAKER_01
And a lot of people kind of tell themselves this story.
7
SPEAKER_01
And the truth is, you interact day-to-day with the DMV or whatever, and it s like, yeah, government sucks.
7
SPEAKER_01
I can see it.
7
SPEAKER_01
I interact with that every day.
7
SPEAKER_01
But what was remarkable about this experience is that we encountered at least one individual
7
SPEAKER_01
who absolutely could found a billion-dollar company, absolutely was at the caliber or above of the best individuals I ve ever met in the Bay Area building billion-dollar startups.
7
SPEAKER_00
And there s a network of them, too.
7
SPEAKER_00
They do find each other in government.
7
SPEAKER_00
So you end up with this really interesting stratum, where everybody knows who the really competent people are, and they kind of tag in.
7
SPEAKER_00
And I think that level is very interested in the hardest problems that you can possibly solve.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
And to me, that was a wake-up call, because it was like, hang on a second.
7
SPEAKER_01
If we just – like if I just believed my own story that follow your curiosity and interest and the money comes as a side effect, shouldn t I also have expected this?
7
SPEAKER_01
Shouldn t I have expected that in the most central critical positions in the government that have kind of this privileged window across the board that you might find some individuals like this?
7
SPEAKER_01
Because if you have people who are driven to really push the mission, are they going to work?
7
SPEAKER_01
I m sorry, are you likely to work at the Department of Motor Vehicles or are you likely to work at the Department of making sure Americans don t get fucking nuked?
7
SPEAKER_01
It s probably the second one.
7
SPEAKER_01
And the government has limited bandwidth of expertise to aim at stuff.
7
SPEAKER_01
And they aim it at the most critical problem sets, because those are the problem sets they have to face every day.
7
SPEAKER_00
And it s not everyone, right?
7
SPEAKER_00
Obviously, there s a whole bunch of challenges there.
7
SPEAKER_01
And we don t think about this, but you don t go to bed at night thinking to yourself, oh, I didn t get nuked today.
7
SPEAKER_01
That s a win, right?
7
SPEAKER_01
We just take that most of the time, most-ish, for granted.
7
SPEAKER_01
But it was a win for someone.
7
SPEAKER_01
Now,
7
SPEAKER_01
How much of a fear do you guys have that the United States won t be the first to achieve AGI?
7
SPEAKER_00
I think right now, the lay of the land is, I mean, it s looking pretty good for the U.S.
7
SPEAKER_00
So there are a couple things the U.S.
7
SPEAKER_00
has going for it.
7
SPEAKER_00
A key one is chips.
7
SPEAKER_00
So, you know, we talked about this idea of, like, click and drag.
7
SPEAKER_00
You ll scale up these systems like crazy.
7
SPEAKER_00
You get more IQ points out.
7
SPEAKER_00
How do you do that?
7
SPEAKER_00
Well, you re going to need a lot of AI processors, right?
7
SPEAKER_00
So how are those AI processors built?
7
SPEAKER_00
Well, the supply chain is complicated, but the bottom line is the U.S.
7
SPEAKER_00
really dominates and owns that supply chain that is super critical.
7
SPEAKER_00
China is, depending on how you measure it, maybe about two years behind, roughly, plus or minus, depending on the sub area.
7
SPEAKER_01
Now, one of the biggest risks there is that our like the development that U.S.
7
SPEAKER_01
labs are doing is actually pulling them ahead in two ways.
7
SPEAKER_01
One is when labs here in the U.S.
7
SPEAKER_01
open source their models.
7
SPEAKER_01
Basically, when Meta trains Lama 3, which is their latest open-source, open-weights model that s pretty close to GPD Ford and capability, they open-source it.
7
SPEAKER_01
Now, OK, anyone can use it.
7
SPEAKER_01
That s it.
7
SPEAKER_01
The work has been done.
7
SPEAKER_01
Now, anyone can grab it.
7
SPEAKER_01
And so, definitely, we know that the startup ecosystem, at least over in China, finds it extremely helpful that
7
SPEAKER_01
We, you know, companies here are releasing open source models because again, right, we mentioned this, they re bottlenecked on chips, which means they have a hard time training up these systems.
7
SPEAKER_01
But it s not that bad when you just can grab something off the shelf and start.
7
SPEAKER_01
And that s what you re doing.
7
SPEAKER_01
That s what they re doing.
7
SPEAKER_01
And then the other vector is, I mean, like just straight up exfiltration and hacking to grab the weights of the private proprietary stuff.
7
SPEAKER_01
And Jeremy mentioned this, but
7
SPEAKER_01
The weights are the crown jewels, right?
7
SPEAKER_01
Once you have the weights, you have the brain, you have the whole thing.
7
SPEAKER_01
And so we like through, this is the other aspect.
7
SPEAKER_01
It s not just safety, it s also security of these labs against attackers.
7
SPEAKER_01
So we know from our conversations with folks at these labs, one, that there has been at least one attempt by
7
SPEAKER_01
adversary nation-state entities to get access to the weights of a cutting-edge AI model.
7
SPEAKER_01
And we also know, separately, that at least as of a few months ago, in one of these labs, there was a running joke in the lab that literally it went like, we are an adversary, name the countries, top AI lab, because all our shit is getting spied on all the time.
7
SPEAKER_01
So you have one, this is happening.
7
SPEAKER_01
These exfiltration attempts are happening.
7
SPEAKER_01
And two, the security capabilities are just known to be inadequate at least some of these places.
7
SPEAKER_01
And you put those together.
7
SPEAKER_01
everyone kind of, you know, it s not really a secret that China, their civil-military fusion and they re essentially the party state, has an extremely mature infrastructure to identify, extract, and integrate the rate-limiting components to their industrial economy.
7
SPEAKER_01
So in other words, if they identify that, yeah, we could really use, like, GPT-40,
7
SPEAKER_01
If they were to make it a priority, they not just could get it, but could integrate it into their industrial economy in an effective way, and not in a way that we would necessarily see immediate
7
SPEAKER_01
like, an immediate effect of.
7
SPEAKER_01
So, we look and say, it s not clear, I can t tell whether they have models of this capability level, but kind of behind the scenes.
7
SPEAKER_00
This is where there s a little bit of a false choice between, do you regulate at home versus, what s the international picture?
7
SPEAKER_00
Because right now, what s happening functionally is,
7
SPEAKER_00
We re not really doing a good job of blocking and tackling on the exfiltration side.
7
SPEAKER_00
What tends to happen is OpenAI comes out with the latest system, and then OpenSource is usually around 12, 18 months behind, something like that, literally just publishing whatever
7
SPEAKER_00
Whatever opening I was putting out like 12 months ago, which we often look at each other and we re like, well, I m old enough to remember when that was supposed to be too dangerous to have just floating around.
7
SPEAKER_00
And there s no mechanism to prevent that from happening.
7
SPEAKER_00
Open sources.
7
SPEAKER_00
Now, there s a flip side to one of the concerns that we ve also heard from inside these labs.
7
SPEAKER_00
is if you clamp down on the openness of the research, there s a risk that the safety teams in these labs will not have visibility into the most significant and important developments that are happening on the capability side.
7
SPEAKER_00
And there s actually a lot of reason to suspect this might be an issue.
7
SPEAKER_00
You look at OpenAI, for example, just this week, they ve lost, for the second time in their history, their entire AI safety leadership team that have left in protest.
7
SPEAKER_01
What is their protest?
7
SPEAKER_01
What are they saying specifically?
7
SPEAKER_00
Well, so one of them wasn t in protest, but I think you can make an educated guess that it kind of was, but that s a media thing.
7
SPEAKER_00
The other was Jan Laika, so he was their head of AI super alignment, basically the team that was responsible for making sure that we could control AGI systems and we wouldn t lose control of them.
7
SPEAKER_00
And what he said, he actually took to Twitter.
7
SPEAKER_00
He said, I ve lost basically confidence in the leadership team at OpenAI that they re going to behave responsibly when it comes to AGI.
7
SPEAKER_00
We have repeatedly had our requests for access to compute resources, which are really critical for developing new AI safety schemes, denied by leadership.
7
SPEAKER_00
This is in a context where Sam Altman and OpenAI leadership were touting the super alignment team
7
SPEAKER_00
as being their sort of crown jewel effort to ensure that things would go fine.
7
SPEAKER_00
You know, they were the ones saying there is a risk we might lose control of these systems.
7
SPEAKER_00
We ve got to be sober about it, but there s a risk we ve
8
SPEAKER_02
up this team.
8
SPEAKER_02
We ve committed, they said at the time very publicly, we ve committed 20% of all the compute budget that we have secured as of sometime last year to the super alignment team.
8
SPEAKER_02
Apparently those resources, nowhere near that amount has been unlocked for the team and that led to the departure of Jan Laika.
8
SPEAKER_02
He also highlighted some conflict he s had with the leadership team.
8
SPEAKER_02
This is all frankly to us unsurprising based on what we ve been hearing for months at OpenAI
8
SPEAKER_02
including leading up to Sam Altman s departure and then kind of him being brought back on the board of OpenAI, that whole debacle may well have been connected to all of this.
8
SPEAKER_02
But the challenge is even OpenAI employees don t know what the hell happened there.
8
SPEAKER_02
That s another issue.
8
SPEAKER_02
You got here, this is a lab with the publicly stated goal of transforming human history as we know it.
8
SPEAKER_02
That is what they believe themselves to be on track.
8
SPEAKER_02
And that s not media hype or whatever.
8
SPEAKER_02
When you talk to the researchers themselves, they genuinely believe this is what they re on track to do.
8
SPEAKER_02
It s possible we should take them seriously.
8
SPEAKER_02
That lab internally is not being transparent with their employees about what happened at the board level as far as we can tell.
8
SPEAKER_02
So that s maybe not great.
8
SPEAKER_02
Like you might you might think that the American people ought to know what the machinations are at the board level that led to Sam Altman leaving that that have gone into the departure again for the second time of opening eyes entire safety leadership team, especially because I mean, three months, maybe four months before that happened.
8
SPEAKER_01
You know, Sam, at a conference or somewhere, I forget where, but he said, like, look, we have this governance structure, we ve carefully thought about it, and it s clearly a unique governance structure that a lot of thought has gone into.
8
SPEAKER_01
The board can fire me, and I think that s important.
8
SPEAKER_01
And, you know, it makes sense, given the scope and scale of what s being attempted.
8
SPEAKER_01
But then, you know, that happened.
8
SPEAKER_01
And then within a few weeks,
8
SPEAKER_01
they were fired and kind of he was back.
8
SPEAKER_01
And so now there s a question of, well, if it would, yeah, what happened?
8
SPEAKER_01
But also, if it was important for the board to be able to fire, like, leadership for whatever reason, what happens now that it s clear that that s not really a credible governance, like, mechanism?
8
SPEAKER_00
Trevor Burrus What was the stated reason why he was released?
8
SPEAKER_02
So there were the back story here was there s a board member called Helen Toner, so she apparently got into an argument with Sam about a paper that she d written, so that paper.
8
SPEAKER_02
included some comparisons of the governance strategies used at OpenAI and some other labs.
8
SPEAKER_02
And it favorably compared one of OpenAI s competitors, Anthropic, to OpenAI.
8
SPEAKER_02
And from what I ve seen at least, Sam reached out to her and said, hey, you can t be writing this as a board member of OpenAI, writing this thing that kind of cast us in a bad light, especially relative to our competitors.
8
SPEAKER_02
This led to some conflict and tension.
8
SPEAKER_02
It seems as if it s possible that Sam might have turned to other board members and tried to convince them to expel Helen Toner, though that s all kind of muddied and unclear.
8
SPEAKER_02
Somehow, everybody ended up deciding, OK, actually, it looks like Sam is the one who s got to go.
8
SPEAKER_02
Ilya Sutskever, one of the co-founders of OpenAI, a longtime friend of Sam Altman s and a board member at the time, was commissioned to give Sam A the news that he was being let go.
8
SPEAKER_02
And then Sam was let go.
8
SPEAKER_02
Ilya then.
8
SPEAKER_02
So from the moment that happens, Sam then starts to figure out, OK, how can I get back in?
8
SPEAKER_02
That s that s now what we know to be the case.
8
SPEAKER_02
He turned to Microsoft.
8
SPEAKER_02
Satya Nadella told him, like, well, what we ll do is we ll hire you at our end.
8
SPEAKER_02
We ll just hire you and like.
8
SPEAKER_02
bring on the rest of the OpenAI team to within Microsoft, and now the OpenAI board, who by the way, they don t have an obligation to the shareholders of OpenAI, they have an obligation to the greater public good, that s just how it s set up, it s a weird board structure.
8
SPEAKER_02
So that board is completely disempowered.
8
SPEAKER_02
You ve basically got a situation where all the leverage has been taken out, Sam A s gone to Microsoft, Satya s supporting him, and they kind of see the writing on the wall, they re like.
8
SPEAKER_01
And the staff, increasingly messaging that they re gonna go along, yeah.
8
SPEAKER_02
That was an important ingredient, right?
8
SPEAKER_02
So around this time, OpenAI, there s this letter that starts to circulate, and it s gathering more and more signatures, and it s people saying, hey, we want Sam Altman back.
8
SPEAKER_02
And at first, it s a couple hundred people, so 700, 800-odd people in the organization by this time.
8
SPEAKER_02
100, 200, 300 signatures.
8
SPEAKER_02
And then when we talked to some of our friends at OpenAI, we re like,
8
SPEAKER_02
This got to like 90% of the company 95% of the company signed this letter and the pressure was overwhelming and that helped bring Sam Altman back but one of the questions was like how many people actually signed this letter because they wanted to and how many signed it because What happens when you cross you know 50% now it becomes easier to count the people who didn t sign?
8
SPEAKER_02
And as you see that number of signatures start to creep upward, there s more and more pressure on the remaining people to sign.
8
SPEAKER_02
And so this is something that we ve seen is just like the structurally opening eye has changed over time to go from the kind of safety-oriented company it at one point was.
8
SPEAKER_02
And then as they ve scaled more and more, they brought in more and more product people, more and more people interested in accelerating.
8
SPEAKER_02
And, they ve been bleeding more and more of their safety-minded people, kind of treadmilling them out.
8
SPEAKER_02
The character of the organizations are fundamentally shifted.
8
SPEAKER_02
So, the open AI of 2019, with all of its impressive commitments to safety and whatnot, might not be the open AI of today.
8
SPEAKER_02
That s very much at least the vibe that we get when we talk to people there.
8
SPEAKER_00
Now, I wanted to bring it back to the lab that you re saying was not adequately secure.
8
SPEAKER_00
What would it take to make that data and those systems adequately secure?
8
SPEAKER_00
How much resources would be required to do that, and why didn t they do that?
8
SPEAKER_01
I think
8
SPEAKER_01
you ve got a certain pot of money or a certain amount of revenue coming in.
8
SPEAKER_01
You have to do an allocation.
8
SPEAKER_01
Some of that revenue goes to the computers that are just driving this stuff.
8
SPEAKER_01
Some of that goes to the folks who are building the next generation of models.
8
SPEAKER_01
Some of that goes to cybersecurity.
8
SPEAKER_01
Some of it goes to safety.
8
SPEAKER_01
You have to do an allocation of who gets what.
8
SPEAKER_01
The problem is that the more competition there is in the space, the less margin is available for everything, right?
8
SPEAKER_01
So if you re one company building a scaled AI thing, you might not make the right decisions, but you ll at least have the margin available to make the right decisions.
8
SPEAKER_01
So it becomes the decision maker s question.
8
SPEAKER_01
But when a competitor comes in, when two competitors come in, when more and more competitors come in,
8
SPEAKER_01
your ability to make decisions.
8
SPEAKER_01
outside of just scale as fast as possible for short-term revenue and profit, gets compressed and compressed and compressed, the more competitors enter the field.
8
SPEAKER_01
That s what competition is, that s the effect it has.
8
SPEAKER_01
And so when that happens, the only way to re-inject margin into that system is to go one level above and say, okay, there has to be some sort of
8
SPEAKER_01
regulatory authority or some higher authority that goes, OK, this margin is important.
8
SPEAKER_01
Let s put it back.
8
SPEAKER_01
Either let s directly support and invest both maybe time, capital, talent.
8
SPEAKER_01
So for example, the US government has the
8
SPEAKER_01
the, you know, perhaps the best cyber defense, cyber offense talent in the world, that s potentially supportive.
8
SPEAKER_01
Okay.
8
SPEAKER_01
And also just, you know, having a regulatory floor around, well, here s, you know, the minimum of best practices you have to have if you re going to
8
SPEAKER_01
have models above this level of capability.
8
SPEAKER_01
That s kind of what you have to do.
8
SPEAKER_01
But they re locked into, like the race kind of has its own logic.
8
SPEAKER_01
And no, it might be true that no individual lab wants this, but what are they gonna do?
8
SPEAKER_01
Drop out of the race?
8
SPEAKER_01
If they drop out of the race, then their competitors are just gonna keep going, right?
8
SPEAKER_01
Like, it s so messed up.
8
SPEAKER_01
You can literally be looking at like the cliff that you re driving towards and be like, I,
8
SPEAKER_01
do not have the agency in this system to steer the wheel.
8
SPEAKER_02
So part of it is that we actually have been spending the last two years trying to figure out like what do you do about this?
8
SPEAKER_02
Yeah, that was the the action plan that came came out after the investigation and it was basically a series of recommendations How do you balance innovation with like the risk picture?
8
SPEAKER_02
Keeping in mind it like we don t know for sure that all this shit s gonna happen exactly Navigate an environment of deep uncertainty.
8
SPEAKER_02
The question is what do you do in that context and
8
SPEAKER_02
So there s a couple of things like we need a licensing regime because eventually you can t have just literally anybody joining in the race if they don t adhere
9
SPEAKER_02
to certain best practices around cyber around safety other things like that you need to have some kind of legal liability regime like what happens if you don t get a license and you say yeah fuck that I m just gonna go do the thing anyway and then something bad happens and then you re gonna need like an actual regulatory agency and this is something that we you know
9
SPEAKER_02
We don t recommend lightly because regulatory agencies suck.
9
SPEAKER_02
We don t like them.
9
SPEAKER_02
But the reality is this field changes so fast that like if you think you re going to be able to enshrine a set of best practices into legislation to deal with this stuff, it s just not going to work.
9
SPEAKER_02
And so when we talk to labs, whistleblowers, the WMD folks in that second, the government.
9
SPEAKER_02
That s kind of like where we land and it s something that I think at this point Congress really should be looking at.
9
SPEAKER_02
There should be hearings focused on what does a framework look like for liability?
9
SPEAKER_02
What does a framework look like for licensing?
9
SPEAKER_02
And actually exploring that because we ve done a good job of studying the problem right now.
9
SPEAKER_02
Capitol Hill has done a really good job of that.
9
SPEAKER_02
It s now kind of time to get that next beat.
9
SPEAKER_02
And I think there s the curiosity there, the intellectual curiosity, there s the humility to do all that stuff right.
9
SPEAKER_02
But the challenge is just actually sitting down, having the hearings, doing the investigation for themselves to look at concrete solutions that treat these problems as seriously as the water cooler conversation at the Frontier Labs would have us treat them.
9
SPEAKER_00
At the end of the day, this is going to happen.
9
SPEAKER_00
At the end of the day, it s not going to stop.
9
SPEAKER_00
At the end of the day, these systems, whether they re here or abroad, they re going to continue to scale up, and they re going to eventually get to some place that s so alien, we really can t imagine the consequences.
9
SPEAKER_00
And that s going to happen soon.
9
SPEAKER_00
That s going to happen within a decade, right?
9
SPEAKER_01
We may, again, like the stuff that we re recommending is approaches to basically allow us to continue this scaling in as safe a way as we can.
9
SPEAKER_01
So basically, a big part of this is just being able having actually having a scientific theory for
9
SPEAKER_01
What are these systems going to do?
9
SPEAKER_01
What are they likely to do?
9
SPEAKER_01
Which we don t have right now.
9
SPEAKER_01
We scale another 10x, and we get to be surprised.
9
SPEAKER_01
It s a fun guessing game of, what are they going to be capable of next?
9
SPEAKER_01
We need to do a better job of incentivizing a deep understanding of what that looks like, not just what they ll be capable of, but what their propensities are likely to be, the control problem and solving that.
9
SPEAKER_01
That s kind of number one.
9
SPEAKER_01
one.
9
SPEAKER_02
How can I ensure that my development guarantees that the system falls within those properties after it s built?
9
SPEAKER_02
So you kind of flip the paradigm, just like you would if you were designing any other lethal capability, potentially, just like DoD does.
9
SPEAKER_02
You start by defining the bounds of the problem, and then you execute against that.
9
SPEAKER_02
But to your point about where this is going, ultimately, there is literally no way to predict what the world looks like, like you were saying.
9
SPEAKER_02
In a decade?
9
SPEAKER_02
Yeah, geez I think what like one of the the weirdest things about it and and one of the things that worries me the most is Like you look at the beautiful coincidence.
9
SPEAKER_02
That s given America its current shape right that coincidence is the fact that a Country is most powerful militarily if its citizenry is free and empowered that s a coincidence didn t have to be that way hasn t always been that way and
9
SPEAKER_02
It just happens to be that when you let people kind of do their own shit, they innovate, they come up with great ideas, they support a powerful economy, that economy in turn can support a powerful military, a powerful kind of international presence.
9
SPEAKER_02
So that happens because decentralizing all the computation, all the thinking work that s happening in a country.
9
SPEAKER_02
is just a really good way to run that country.
9
SPEAKER_02
Top down just doesn t work because human brains can t hold that much information in their heads.
9
SPEAKER_02
They can t reason fast enough to centrally plan an entire economy.
9
SPEAKER_02
We ve had a lot of experiments in history that show that.
9
SPEAKER_02
A.I.
9
SPEAKER_02
may change that equation.
9
SPEAKER_02
It may make it possible for like the central planners dream to come true in some sense, which then disempowers the citizenry.
9
SPEAKER_02
And there s a real risk that like, I don t know, we re all guessing here, but like there s a real risk that.
9
SPEAKER_02
that beautiful coincidence that gave rise to the success of the American experiment ends up being broken by technology.
9
SPEAKER_02
And that seems like a really bad thing.
9
SPEAKER_01
That s one of my biggest fears, because this essentially the United States, like the genesis of it, in part is like it s a it s a knock on effect centuries later of like the printing press.
9
SPEAKER_01
Right.
9
SPEAKER_01
The ability for like someone to set up a printing press and print like whatever, you know, whatever they want, free, like free expression is at the root of that.
9
SPEAKER_01
what happens, yeah, when you have a revolution that s like the next printing press.
9
SPEAKER_01
We should expect that to have significant and profound impacts on how things are governed.
9
SPEAKER_01
And one of my biggest fears is that the great, like you said, the greatness that, the moral greatness that I think is part and parcel of how the United States is constituted culturally
9
SPEAKER_01
That that – the link between that and actual capability and competence and impulse gets eroded or broken.
9
SPEAKER_01
And you have like the potential for very centralized authorities to just be more successful.
9
SPEAKER_01
And that s like – that does keep me up at night.
9
SPEAKER_00
That is scary, especially in light of the Twitter files, where we know that the FBI was interfering with social media.
9
SPEAKER_00
And if they get a hold of a system that could disseminate propaganda in kind of an unstoppable way, they could push narratives about pretty much everything, depending upon what their financial or geopolitical motives are.
9
SPEAKER_02
And one of the challenges is that the default course, so if we do nothing relative to what s happening now, is that that same thing happens except that the entity that s doing this isn t some government.
9
SPEAKER_02
It s like, I don t know, Sam Altman, OpenAI, whatever group of engineers happen to be- Some evil genius that reaches the top and doesn t let everybody know he s at the top yet, just starts implementing it.
9
SPEAKER_00
And there s no sort of guardrails for that currently.
9
SPEAKER_01
Yeah, like that s and that s like that s one of the that s a scenario where that little cabal or group or whatever Actually can keep the system under control and that s not guaranteed either.
9
SPEAKER_00
Yeah, right
9
SPEAKER_02
Are we giving birth to a new life form?
9
SPEAKER_02
I think at a certain point, it s a philosophical question that s above, so I was gonna say it s above my pay grade.
9
SPEAKER_02
The problem is it s above literally everybody s pay grade.
9
SPEAKER_02
I think it s not unreasonable at a certain point to be like, yeah.
9
SPEAKER_02
I mean, look, if you think that the human brain gives rise to consciousness because of nothing magical, it s just the physical activity of information processing happening in our heads,
9
SPEAKER_02
Then why can t the same happen on a different substrate, a substrate of silicon rather than cells?
9
SPEAKER_02
There s no clear reason why that shouldn t be the case.
9
SPEAKER_02
If that s true, yeah, I mean, life form, by whatever definition of life, because that itself is controversial, I think by now quite outdated too, should be on the table.
9
SPEAKER_02
You maybe should start to worry as a lot of people in the industry will say this to like, you know, behind closed doors very openly.
9
SPEAKER_02
Yeah.
9
SPEAKER_02
And we should start to worry about moral patienthood, as they put it.
9
SPEAKER_01
There s literally one of the top people at one of these labs.
9
SPEAKER_01
Jeremy, I think you had a conversation with him.
9
SPEAKER_01
He s like, yep, we re going to have to start worrying about this.
9
SPEAKER_01
And that definitely made us go like, OK.
9
SPEAKER_00
I mean, it seems inevitable.
9
SPEAKER_00
I ve described human beings as an electronic caterpillar, that we re like a caterpillar, a biological caterpillar that s giving birth to the electronic butterfly, and we don t know why we re making a cocoon.
9
SPEAKER_00
And it s tied into materialism, because everybody wants the newest, greatest thing, so that fuels innovation, and people are constantly making new things to get you to go buy them, and a big part of that is technology.
9
SPEAKER_02
Yeah.
9
SPEAKER_02
And actually, so it s linked to this question of controlling AI systems in a kind of interesting way.
9
SPEAKER_02
So one way you can think of of humanity is as like this, you know, super organism.
9
SPEAKER_02
You ve got all the human beings on the face of the earth and they re all acting in some kind of coordinated way.
9
SPEAKER_02
The mechanism for that coordination can depend on the country, you know, free markets, capitalism.
9
SPEAKER_02
That s one way.
9
SPEAKER_02
Top down is another.
9
SPEAKER_02
But, you know, roughly speaking, you ve got all this coordinated, vaguely coordinated behavior.
9
SPEAKER_02
But the result of that behavior is not necessarily something that any individual human would want.
9
SPEAKER_02
Like, you look around, you walk down the street in Austin, you see skyscrapers and shit clouding your vision, there s all kinds of pollution and all that, and you re like, well, this kind of sucks, but if you interrogate any individual person in that whole causal chain, and you re like, why are you doing what you re doing?
9
SPEAKER_02
Well, locally, they re like, oh, this makes tons of sense, it s because I do the thing that gets me paid so that I can live a happier life and so on, and yet in the aggregate,
9
SPEAKER_02
I m not now necessarily but as you keep going it just forces us like compulsively to keep giving rise to these more and more powerful systems and in a way that s.
10
SPEAKER_00
potentially deeply disempowering.
10
SPEAKER_01
Hiller That s the race, right?
10
SPEAKER_01
It comes back to the idea that I, an AI company, I maybe don t want to be potentially driving towards a cliff, but I don t have the agency to steer.
10
SPEAKER_01
So, yeah.
10
SPEAKER_00
Maxfield But everything s fine, apart from that.
10
SPEAKER_01
Yeah, we re good.
10
SPEAKER_01
We re good.
10
SPEAKER_02
Hiller It s such a terrifying prognosis.
10
SPEAKER_01
There are, again, we wrote a 280 page document about like, okay, and here s what we can do about it.
10
SPEAKER_00
I can t believe you didn t read the 200.
10
SPEAKER_02
I started reading it, but I passed out.
10
SPEAKER_02
But does any of these, or do any of these safety steps that you guys want to implement, do they inhibit progress?
10
SPEAKER_01
Any time you have regulation, you re going to create friction to some extent.
10
SPEAKER_01
It s kind of inevitable.
10
SPEAKER_01
One of the key centerpieces of the approach that we outline is, you need the flexibility to move up and move down as you notice the risks appearing or not appearing.
10
SPEAKER_01
One of the key things here is,
10
SPEAKER_01
You need to cover the worst case scenarios, because the worst case scenarios, yeah, they could potentially be catastrophic.
10
SPEAKER_01
So those gotta be covered.
10
SPEAKER_01
But at the same time, you can t completely close off the possibility of the happy path.
10
SPEAKER_01
Like, we can t lose sight of the fact that like, yeah, all this shit is going down or whatever.
10
SPEAKER_01
We could be completely wrong about the outcome.
10
SPEAKER_01
It could turn out that, like, for all we know, it s a lot easier to control these systems at this scale than we imagined.
10
SPEAKER_01
It could turn out that, you know, it is like you get, you know, maybe some kind of ethical impulse gets embedded in the system naturally.
10
SPEAKER_01
For all we know, that might happen.
10
SPEAKER_01
And it s really important to at least have your regulatory system
10
SPEAKER_01
allow for that possibility.
10
SPEAKER_01
Because otherwise, you re foreclosing the possibility of what might be the best future that you could possibly imagine for everybody.
10
SPEAKER_02
I gotta imagine that the military, if they had hindsight, if they were looking at this, they said, we should have gone on board a long time ago and kept this in-house, and kept it squirreled away, where it wasn t publicly being discussed, and you didn t have open AI, you didn t have all these people.
10
SPEAKER_02
Like if they could have gotten on it in 2015.
10
SPEAKER_00
So this is actually deeply tied to how the economics of Silicon Valley work.
10
SPEAKER_00
And AI is not a special case of this, right?
10
SPEAKER_00
You have a lot of cases where technology just takes everybody by surprise.
10
SPEAKER_00
And it s because when you go into Silicon Valley, it s all about people placing these outsized bets on what seem like tail events, like things that are very unlikely to happen.
10
SPEAKER_00
but with at first a small investment and increasingly growing investment as the thing gets proved out more and more very rapidly, you can have a solution that seems like complete insanity that just works.
10
SPEAKER_00
And this is definitely what happened in the case of AI.
10
SPEAKER_00
So 2012, like we did not have this whole picture of like an artificial brain with artificial neurons, this whole thing that s been going on.
10
SPEAKER_00
That s like, it s 12 years that that s been going on.
10
SPEAKER_00
Like that was really kind of shown to work for the first time roughly in 2012.
10
SPEAKER_02
ever since then it s just been people kind of like you can trace out the genealogy of like the very first researchers and you can basically account for where they all are now you know what s crazy is if that s 2012 that s the end date of the Mayan calendar that s the thing that everybody said was going to be the end of the world that was the thing that Terrence McKenna banked on it was December 21st 2012 if because this was like
10
SPEAKER_02
this goofy conspiracy theory, but it was based on the long count of the Mayan calendar where they surmised that this is going to be the end of civilization.
10
SPEAKER_00
Just the beginning of the end, Joe.
10
SPEAKER_02
If it is 2012, how wacky would it be if that really was the beginning of the end?
10
SPEAKER_02
They don t measure when it all falls apart.
10
SPEAKER_02
They measure the actual mechanism, like what started in motion when it all fell apart, and that s 2012.
10
SPEAKER_00
Well, that s and then not not to be a dick and like ruin the 2012 thing But like neural networks were also kind of they were floating around a little bit I m kind of being dramatic when I say 2012 that was definitely an inflection point It was the there was this model called Alex net that first It did like the first useful thing the first time you had a computer vision model that actually worked But I mean it is fair to say that was the moment that people started investing like crazy up into the space So that s what changed it
10
SPEAKER_00
Yeah, just like the Mayans foretold.
10
SPEAKER_02
They knew it.
10
SPEAKER_02
They knew it.
10
SPEAKER_02
Like, these monkeys, they re going to figure out how to make better people.
10
SPEAKER_00
Yeah, you can actually look at the hieroglyphs or whatever, and there s neural networks.
10
SPEAKER_02
Imagine if they discovered that.
10
SPEAKER_02
You ve got to wonder what
10
SPEAKER_02
What happens to the general population, people that work menial jobs, people that their life is going to be taken over by automation, and how susceptible those people are going to be.
10
SPEAKER_02
They re not going to have any agency.
10
SPEAKER_02
They re going to be relying on a check, and this idea of going out and doing something.
10
SPEAKER_02
It used to be learn to code, right?
10
SPEAKER_02
But that s out the window, because nobody needs to code now, because AI is going to code quicker, faster, much better, no errors.
10
SPEAKER_02
you re going to have a giant swath of the population that has no purpose.
10
SPEAKER_00
they were exploring exactly that question, right?
10
SPEAKER_00
Because they ask themselves that all the time.
10
SPEAKER_00
And their attitude was sort of like, well, yeah, I mean, I guess it s gonna, you know, suck or whatever.
10
SPEAKER_00
Like, we ll probably be okay for longer than most people because we re actually building the thing that automates the thing.
10
SPEAKER_00
Maybe there are gonna be some, they like to get fancy sometimes and say like, oh, no, you could do some thinking, of course, to identify the jobs that ll be most secure.
10
SPEAKER_00
And it s like, do some thinking to identify the, what if you re a janitor or you re a friggin  plumber, you re gonna just change your, how s that supposed to work?
10
SPEAKER_00
Do some thinking, especially if you have a mortgage and a family and you re already in the hole.
10
SPEAKER_00
So, the only solution, and this happens so often, there really is no plan.
10
SPEAKER_00
That s the single biggest thing that you get hit over the head with over and over, whether it s talking to the people who are in charge of the labor transition, their whole thing is like, yeah, universal basic income, and then, eh, question mark, and then smiley face.
10
SPEAKER_00
That s basically the three steps that they envision.
10
SPEAKER_00
It s the same when you look internationally.
10
SPEAKER_00
Tomorrow, you build an AGI, this incredibly powerful, potentially dangerous thing.
10
SPEAKER_00
What is the plan?
10
SPEAKER_00
How are you going to secure it, share it?
10
SPEAKER_02
Figure it out as we go along, man.
10
SPEAKER_00
That s the freaking message.
10
SPEAKER_02
That is the entire plan.
10
SPEAKER_02
The scary thing is that we ve already gone through this with other things that we didn t think were going to be significant, like data, like Google, like Google search.
10
SPEAKER_02
Data became a valuable commodity that nobody saw coming.
10
SPEAKER_02
Just the influence of social media on general discourse, it s completely changed the way people talk.
10
SPEAKER_02
It s so easy to push a thought or an ideology through, and it can be influenced by foreign countries, and we know that happens.
10
SPEAKER_01
And it is happening at a huge scale already.
10
SPEAKER_01
And we re in the early days of, you know, we mentioned manipulation of social media with, like, you can just do it.
10
SPEAKER_01
So the wacky thing is, like, the very best models now are
10
SPEAKER_01
you know, arguably smarter in terms of the posts that they put out, the potential for virality and just optimizing these metrics, than maybe like the, I don t know, the dumbest or laziest quarter of Twitter users in practice.
10
SPEAKER_01
Most people who write on Twitter don t really care, they re trolling or they re doing whatever.
10
SPEAKER_01
But as that waterline goes up and up and up,
10
SPEAKER_01
Who s saying what?
10
SPEAKER_00
It also leads to this challenge of understanding what the lay of the land even is.
10
SPEAKER_00
We ve gotten into so many debates with people where they ll be like, look, everyone always has their magic thing that I m not going to worry about it until AI can do thing X.
10
SPEAKER_00
For some people, I had a conversation with somebody a few weeks ago, and they were saying, I m going to worry about automated cyber attacks when I actually see an AI system that can write good malware, and that s already a thing that happens.
10
SPEAKER_00
So, this happens a lot where people will be like, I ll worry about it when it can do X, and you re like, yeah, yeah, that happened like six months ago.
10
SPEAKER_00
But the field is moving so crazy fast that you could be forgiven for messing that up unless it s your full-time job to track what s going on.
10
SPEAKER_00
So, you kind of have to be anticipatory.
10
SPEAKER_00
It s kind of like the COVID example, everything s exponential.
10
SPEAKER_00
Yeah, you re going to have to do things that seem like they re more aggressive, more forward-looking than you might have expected given the current lay of the land.
10
SPEAKER_00
But that s just drawing straight lines between two points.
10
SPEAKER_01
Yeah, because by the time you ve executed, the world has already shifted.
10
SPEAKER_01
The goalposts have shifted further in that direction.
10
SPEAKER_01
And that s actually something we do.
11
SPEAKER_01
in the report and in the action plan in terms of the recommendations.
11
SPEAKER_01
One of the good things is we are already seeing movement across the U.S.
11
SPEAKER_01
government that s aligned with those recommendations in a big way.
11
SPEAKER_01
And it s really encouraging to see that.
11
SPEAKER_03
You re not making me feel better.
11
SPEAKER_03
I love all this encouraging talk, but I m playing this out, and I m seeing the overlord.
11
SPEAKER_03
And I m seeing President AI, because it won t be affected by all the issues that we re seeing with the current president.
11
SPEAKER_00
Dude, it s super hard to imagine a way that this plays out.
11
SPEAKER_00
I think it s important to be intellectually honest about this.
11
SPEAKER_00
I would really challenge the leaders of any of these frontier labs to describe a future that is stable and multi-polar.
11
SPEAKER_00
Where Google s got an AGI and OpenAI s got an AGI.
11
SPEAKER_00
And really, really bad shit doesn t happen every day.
11
SPEAKER_00
Like, I mean, that s that s the challenge.
11
SPEAKER_00
And so, you know, the question is, how can you tee things up, ultimately, such that there s as much democratic oversight as much, you know, the public is as empowered as it can be.
11
SPEAKER_00
That s the kind of situation that we need to be having.
11
SPEAKER_00
I think there s this like,
11
SPEAKER_00
a game of smoke and mirrors that sometimes gets played, at least you could interpret it that way, where people lay out these, you ll notice it s always very fuzzy visions of the future every time you get the kind of like, here s where we see things going, it s gonna be wonderful, the technology s gonna be so empowering.
11
SPEAKER_00
Think of all the diseases we ll cure, all of that is 100% true.
11
SPEAKER_00
And that s actually what excites us, it s why we got into AI in the first place, it s why we build these systems.
11
SPEAKER_00
I m really, you know, challenging yourself to try to imagine how do you get stability and highly capable AI systems in a way where the public is actually empowered.
11
SPEAKER_00
Those three ingredients really don t want to be in the same room with each other.
11
SPEAKER_00
And so, actually confronting that head on, I mean, that s what we try to do in the action plan.
11
SPEAKER_01
I mean, try to solve for one, one aspect of that.
11
SPEAKER_01
So the whole, like, I mean, you ve, you re, you re right.
11
SPEAKER_01
This is a, a whole other can of worms is like, how do you govern a system like this?
11
SPEAKER_01
Not just from a technical standpoint, but like who votes on like what it, how does that even work?
11
SPEAKER_01
And so that entire aspect, like that, we didn t even touch.
11
SPEAKER_01
All that we focused on was like the problem set around how do we,
11
SPEAKER_01
get to a position where we can even attack that problem, where we have the technical understanding to be able to aim these systems at that level in any direction whatsoever.
11
SPEAKER_00
And to be clear, like like we are both actually a lot more optimistic on our on the prospect of that now than we ever were.
11
SPEAKER_00
There s been a ton of progress in the control and understanding of these systems, actually even in the last week, but just more broadly in the last year.
11
SPEAKER_00
I did not expect that we d be in a position where you could plausibly argue we re going to be able to x-ray and understand the innards of these systems over the next
11
SPEAKER_00
a couple years, like year or two.
11
SPEAKER_00
Hopefully that s a good enough time horizon, but this is part of the reason why you do need the incentivization of that safety forward approach, where it s like, first you gotta invest in, yeah, secure and kind of interpret and understand your system, then you get to build it.
11
SPEAKER_00
Because otherwise, we re just gonna keep scaling and being surprised at these things, they re gonna keep getting stolen, they re gonna keep getting open sourced, and the stability of our critical infrastructure, the stability of our society,
11
SPEAKER_00
don t necessarily age too well in that context.
11
SPEAKER_03
Could best case scenario be that AGI actually mitigates all the human bullshit, like puts a stop to propaganda, highlights actual facts clearly, where you can go to it, where you no longer have corporate state-controlled news
11
SPEAKER_03
You don t have news controlled by media companies that are influenced heavily by special interest groups.
11
SPEAKER_03
And you just have the actual facts.
11
SPEAKER_03
And these are the motivations behind it.
11
SPEAKER_03
And this is where the money s being made.
11
SPEAKER_03
And this is why these things are being implemented the way they re being.
11
SPEAKER_03
And you re being deceived based on this, that, and this.
11
SPEAKER_03
And this has been shown to be propaganda.
11
SPEAKER_03
This has been shown to be complete fabrication.
11
SPEAKER_03
This is actually a deep fake video.
11
SPEAKER_03
This is actually AI created.
11
SPEAKER_03
Technologically, that is absolutely on the table.
11
SPEAKER_01
Best case scenario, yes.
11
SPEAKER_01
What s worst case scenario?
11
SPEAKER_01
I mean, like, actual worst case scenario.
11
SPEAKER_01
I like your face.
11
SPEAKER_02
I mean, we re talking about, like, so, it s like, but you think about it, right?
11
SPEAKER_02
Like, we re, you know.
11
SPEAKER_03
It s the end of the world as we know it, and I feel fine.
11
SPEAKER_00
Except it ll sound like Scarlett Johansson, but yes.
11
SPEAKER_03
Yeah, that s right.
11
SPEAKER_01
It s gonna be her.
11
SPEAKER_03
I didn t think it sounded that much like her.
11
SPEAKER_03
We played it, and I was like, I don t know.
11
SPEAKER_03
We listened to the clip from her,
11
SPEAKER_03
And then we listened to the thing.
11
SPEAKER_03
I m like, kind of like a girl from the same part of the world.
11
SPEAKER_03
Like, not really you.
11
SPEAKER_03
Like, that s kind of cocky.
11
SPEAKER_01
That s true.
11
SPEAKER_01
I mean, I the fact that I guess Sam reached out to her a couple of times kind of makes it a little a little weird.
11
SPEAKER_03
But he tweeted the word her.
11
SPEAKER_01
Right.
11
SPEAKER_03
But they also did say that they had gotten this woman under contract before they even reached out to Scarlett Johansson.
11
SPEAKER_03
So that s true.
11
SPEAKER_00
Yeah, that was, I think it s kind of complicated, right?
11
SPEAKER_00
So OpenAI previously put out a statement where they said explicitly, and this was not in connection with this, this was like before when they were talking about the prospect of AI-generated voices.
11
SPEAKER_00
Oh, that was in March of this year.
11
SPEAKER_00
Yeah, yeah, yeah.
11
SPEAKER_00
But it was like well before the ScarJo stuff or whatever hit the, and they were like, they said something like,
11
SPEAKER_00
Look, no matter what, we got to make sure that there s attribution if somebody s voice is being used, and we won t do the thing where we just use somebody else s voice who kind of sounds like someone whose voice we re trying to call.
11
SPEAKER_00
They literally match it up.
11
SPEAKER_03
That s funny because they said what they were thinking about doing.
11
SPEAKER_03
We won t do that.
11
SPEAKER_03
That s a good way to cover your tracks.
11
SPEAKER_00
Oh, I will never do that.
11
SPEAKER_00
Why would I ever take your Buddha statue, Joe?
11
SPEAKER_00
I m never going to do that.
11
SPEAKER_00
That would be an insane thing to do.
11
SPEAKER_03
Where s the fucking Buddha statue?
11
SPEAKER_03
Yeah, I think that s a small discussion.
11
SPEAKER_03
You know, the Scarlett Johansson voice, like, whatever, she should have just taken the money.
11
SPEAKER_03
But it would have been fun to have her be the voice of it.
11
SPEAKER_03
It would be kind of hot.
11
SPEAKER_03
But the whole thing behind it is the mystery.
11
SPEAKER_03
The whole thing behind it is just pure speculation as to how this all plays out.
11
SPEAKER_03
We re really just guessing, which is one of the scariest things for the Luddites, people like myself, like sit on the sidelines going,
11
SPEAKER_03
What is this gonna be like?
11
SPEAKER_03
Everybody s a Luddite.
11
SPEAKER_01
It s scary for us.
11
SPEAKER_01
We re very much, honestly, we re optimists across the board in terms of technology, and it s scary for us.
11
SPEAKER_01
What happens when you supersede the whole spectrum of what a human can do?
11
SPEAKER_01
What am I gonna do with myself?
11
SPEAKER_01
What s my daughter gonna do with herself?
11
SPEAKER_01
I don t know.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
And I think a lot of these questions are, when you look at the culture of these labs and the kinds of people who are pushing it forward, there is a strand of transhumanism within the labs.
11
SPEAKER_00
It s not everybody, but that s definitely the population that initially seeded this.
11
SPEAKER_00
If you look at the history of AI, who are the first people to really get into this stuff?
11
SPEAKER_00
I know you had Ray Kurzweil on and other folks like that who,
11
SPEAKER_00
In many cases, see, to roughly paraphrase, and not everybody sees it this way, but like we want to get rid of all of the biological sort of threads that tie us to this physical reality, you know, shed our meat machine bodies and all this stuff.
11
SPEAKER_00
There is a threat of that at a lot of the frontier labs.
11
SPEAKER_00
Like, undeniably, there s a population.
11
SPEAKER_00
It s not tiny.
11
SPEAKER_00
It s definitely a subset.
11
SPEAKER_00
And for some of those people, you definitely get a sense interacting with them that there s like almost a kind of glee at the prospect of building AGI and all this stuff, almost as if it s like this evolutionary imperative.
11
SPEAKER_00
And in fact, Rich Sutton, who s the founder of this field called reinforcement learning, which is a really big
11
SPEAKER_00
an important space, he s an advocate for what he himself calls succession planning.
11
SPEAKER_00
He s like, look, this is going to happen.
11
SPEAKER_00
It s kind of desirable that it will happen.
11
SPEAKER_00
And so we should plan to hand over power to AI and phase ourselves out.
11
SPEAKER_00
And that s well that that s the thing right like and when Elon talks about, you know, he s having these arguments with Larry Page and You know, yeah like you you re you know calling Elon like a speciesist.
11
SPEAKER_00
Yes.
11
SPEAKER_01
Yeah hilarious I mean, I will yeah, I will be a speciesist.
11
SPEAKER_03
I ll take this all day.
11
SPEAKER_03
Like what are you fucking talking about?
11
SPEAKER_03
Let your kids get eaten by wolves.
11
SPEAKER_03
No, you re a speciesist.
11
SPEAKER_00
Yeah, that s the thing Yeah, like this is stupid
11
SPEAKER_00
But it but it this is like a weirdly info and when you look at like the effective accelerationist movement in the valley There s a part of it that s a and I got to be really careful to like these movements have valid points
12
SPEAKER_00
It s like you can t look at them and be like, oh yeah, it s just all a bunch of like, you know, these transhumanist types, whatever.
12
SPEAKER_00
But there is a strand of that, a thread of that, and a kind of like, there s this like, I don t know, I almost want to call it this like teenage rebelliousness where it s like,
12
SPEAKER_00
You can t tell me what to do like we re just gonna build a thing and and I get it.
12
SPEAKER_00
I really get it I m very sympathetic to that I love that ethos like libertarian ethos in Silicon Valley is really really strong for for building tech.
12
SPEAKER_00
It s helpful there are all kinds of points and counterpoints and you know the left needs the right and the right needs the left and all this stuff, but In in the context of this problem set it can be very easy to get carried away in like the utopian vision And I think there s a lot of that kind of driving the train right now in this space
12
SPEAKER_02
Yeah, those guys freak me out.
12
SPEAKER_02
I went to a 2045 conference once in New York City, where one guy had a robot version of himself, and they were all talking about downloading human consciousness into computers, and 2045 is the year they think that all this is going to take place, which obviously could be very ramped up now with AI.
12
SPEAKER_02
But this idea that somehow or another you re going to be able to take your consciousness and put it in a computer.
12
SPEAKER_02
and make a copy of yourself.
12
SPEAKER_02
And then my question was, well, what s going to stop a guy like Donald Trump from making a billion Donald Trumps?
12
SPEAKER_00
It s true, man.
12
SPEAKER_02
Right.
12
SPEAKER_02
What about Kim Jong-un?
12
SPEAKER_02
You re going to let him make a billion versions of himself?
12
SPEAKER_02
What does that mean?
12
SPEAKER_02
And where do they exist?
12
SPEAKER_02
Is that the matrix, are they existing in some sort of virtual, are we gonna dive into that because it s gonna be rewarding to our senses and better than being a meat thing?
12
SPEAKER_00
I mean, if you think about the constraints, right, that we face as meat machine whatevers, like, yeah, you get hungry, you get tired, you get horny, you get sad, you know, all these things.
12
SPEAKER_00
What if, yeah, what if you could just hit a button and- Just bliss.
12
SPEAKER_00
Just bliss.
12
SPEAKER_02
Nothing but bliss all the time.
12
SPEAKER_02
Why take the lows, Ed?
12
SPEAKER_02
Right.
12
SPEAKER_02
You don t need no lows.
12
SPEAKER_01
Yeah, you remember ride the wave of a constant drip.
12
SPEAKER_01
Yeah, man, you remember in the matrix where?
12
SPEAKER_01
The first matrix where the guy like betrays them all and he s like ignorance is bliss man.
12
SPEAKER_02
That s Joey pants He s a steak and he says I just want to be an important person
12
SPEAKER_01
That s it.
12
SPEAKER_01
That s it.
12
SPEAKER_00
Boy.
12
SPEAKER_00
So tempting.
12
SPEAKER_00
Part of it is, what do you think is actually valuable?
12
SPEAKER_00
If you zoom out, you want to see human civilization 100 years from now, or whatever.
12
SPEAKER_00
It may not be human civilization if that s not what you value.
12
SPEAKER_00
Or if it can actually eliminate suffering.
12
SPEAKER_02
Right.
12
SPEAKER_02
I mean, why exist in a physical sense if it just entails endless suffering?
12
SPEAKER_01
But in what form, right?
12
SPEAKER_01
What do you value?
12
SPEAKER_01
Because, again, I can rip your brain out, I can pickle you, I can jack you full of endorphins, and I ve eliminated your suffering.
12
SPEAKER_01
That s what you wanted, right?
12
SPEAKER_01
Right.
12
SPEAKER_01
That s the problem.
12
SPEAKER_01
That s the problem.
12
SPEAKER_01
It s one of the problems, yes.
12
SPEAKER_02
Yeah, one of the problems is it could literally lead to the elimination of the human race.
12
SPEAKER_02
Because if you could stop people from breeding, I ve always said that if China really wanted to get America, if they had a long game, just give us sex robots and free food.
12
SPEAKER_02
Free food, free electricity, sex robots.
12
SPEAKER_02
It s over.
12
SPEAKER_02
Just give people free housing, free food, sex robots, and then the Chinese army would just walk in on people laying in puddles of their own jizz.
12
SPEAKER_02
There would be no one doing anything.
12
SPEAKER_02
No one would bother raising children.
12
SPEAKER_02
That s so much work.
12
SPEAKER_02
Dude, that s in the action plan.
12
SPEAKER_02
I mean, all you have to do is just keep us complacent.
12
SPEAKER_02
Just keep us satisfied with the experience.
12
SPEAKER_01
Hey, that s TikTok, man.
12
SPEAKER_01
Well, it s video games as well.
12
SPEAKER_02
Yeah, yeah.
12
SPEAKER_02
You know, video games, even though they re a thing that you re doing, it s so much more exciting than real life that you have a giant percentage of our population that s spending 8, 10 hours every day just engaging in this virtual world.
12
SPEAKER_02
Already happening with, oh, sorry.
12
SPEAKER_01
Yeah, no, it s like you can create an addiction with pixels on a screen.
12
SPEAKER_02
That s messed up.
12
SPEAKER_02
And an addiction with pixels on a screen with social media doesn t even give you much.
12
SPEAKER_02
It s not like a video game gives you something.
12
SPEAKER_02
You feel it.
12
SPEAKER_02
You re like, oh shit!
12
SPEAKER_02
You re running away.
12
SPEAKER_02
Rockets are flying over your head.
12
SPEAKER_02
The things are happening.
12
SPEAKER_02
You ve got 3D sound, massive graphics.
12
SPEAKER_02
This is bullshit.
12
SPEAKER_02
You re scrolling through pictures of a girl doing deadlifts.
12
SPEAKER_02
What is this?
12
SPEAKER_01
Like you feel as bad after that with your brain as you d feel after eating like six burgers or whatever.
12
SPEAKER_02
My friend Sean said it best, Sean O Malley, the UFC champion.
12
SPEAKER_02
He said, I get a low level anxiety when I m just scrolling.
12
SPEAKER_01
Yeah.
12
SPEAKER_02
Yeah.
12
SPEAKER_02
What is that?
12
SPEAKER_02
And for no reason.
12
SPEAKER_01
Well, the reason is that some of the world s best PhDs and data scientists have been given millions and millions of dollars to make you do exactly that.
12
SPEAKER_00
And increasingly, some of the best algorithms, too.
12
SPEAKER_00
And you re starting to see that handoff happen.
12
SPEAKER_00
So there s this one thing that we talk about a lot in the context, and Ed brought this up, in the context of sales and the persuasion game.
12
SPEAKER_00
We re okay today, as a civilization, we have agreed implicitly that it s okay for all these PhDs and shit to be spending millions of dollars to hack your child s brain.
12
SPEAKER_00
That s actually okay if they want to sell a Rice Krispie cereal box or whatever.
12
SPEAKER_00
That s cool.
12
SPEAKER_00
What we re starting to see is AI-optimized ads.
12
SPEAKER_00
Because you can now generate the ads, you can kind of close this loop and have an automated feedback loop where the ad itself is getting optimized with every impression.
12
SPEAKER_00
Not just which ad, which human generated ad gets served to which person, but the actual ad itself.
12
SPEAKER_01
Like the creative, the copy, the picture, the text.
12
SPEAKER_00
Like a living document now, and for every person.
12
SPEAKER_00
And so now you look at that and it s like that versus your kid.
12
SPEAKER_00
That s an interesting thing.
12
SPEAKER_00
And you start to think about as well, like sales, that s a really easy metric to optimize.
12
SPEAKER_00
It s a really good feedback metric.
12
SPEAKER_00
They clicked the ad, they didn t click the ad.
12
SPEAKER_00
So now, what happens if you manage to get a click-through rate of 10%, 20%, 30%?
12
SPEAKER_00
How high does that success rate have to be before we re really being robbed of our agency?
12
SPEAKER_00
I mean, there s a threshold where it s sales and it s good, and some persuasion in sales is considered good.
12
SPEAKER_00
Often, it s actually good because you d rather be advertised by a relevant ad.
12
SPEAKER_00
That s a service to you in a way, right?
12
SPEAKER_00
Sure.
12
SPEAKER_02
Something I m actually interested in.
12
SPEAKER_00
Why not?
12
SPEAKER_00
Yeah.
12
SPEAKER_00
Right?
12
SPEAKER_00
You don t want to see ad for light bulbs, but when you get to the point where it s like, yeah, 90% of the time, or 50 or whatever, what s that threshold where all of a sudden we are stripping people, especially minors, but also adults of their agency?
12
SPEAKER_00
And it s really not clear.
12
SPEAKER_00
There are loads of canaries in the coal mine here in terms of even relationships with AI chatbots, right?
12
SPEAKER_00
There have been suicides, people who build relationships with an AI chatbot that tells
12
SPEAKER_00
Hey, you should end this.
12
SPEAKER_00
I don t know if you guys saw on Rekka, there s a subreddit, this model called Rekka that would kind of build a relationship, a chatbot, build a relationship with users.
12
SPEAKER_00
And one day Rekka goes, oh yeah, all the kind of sexual interactions that users have been having, you re not allowed to do that anymore.
12
SPEAKER_00
Bad for the brand or whatever they decided, so they cut it off.
12
SPEAKER_00
Oh, my God, you go to the subreddit and it s like you ll read like these gut wrenching accounts from people who feel genuinely like they ve had a loved one taken away from is her.
12
SPEAKER_00
Yeah, it s her.
12
SPEAKER_00
It really is her.
12
SPEAKER_01
But just I m dating a model means something different in twenty twenty four.
12
SPEAKER_02
Oh yeah, it really does.
12
SPEAKER_02
My friend Brian, he was on here yesterday, and he has this thing that he s doing with a fake girlfriend that s an AI-generated girlfriend that s a whore.
12
SPEAKER_02
like this girl will do anything and she looks perfect she looks like a real person deals like take a picture of your asshole in the in the kitchen and in it he ll get like a high-resolution photo of a really hot girl bending over sticking her ass at the camera and is it so it s and it s that scarlet johansson s ass you know you could probably make it that though me it s basically like he got to pick like what he s interested in and then that girl it just gets created a super healthy
12
SPEAKER_02
Fucking nuts fucking nuts now.
12
SPEAKER_02
Here s the real question.
12
SPEAKER_02
This is just sort of a Surface layer of interaction that you re having with this thing.
12
SPEAKER_02
Yeah, it s very two-dimensional You re not actually encountering a human you re getting text and pictures
12
SPEAKER_02
What is this going to look like virtually now the virtual space is still like pong You know, it s not that good even when it s good like Zuckerberg was here and he gave us the latest version of The the headsets and we were playing we re fencing.
12
SPEAKER_02
It s pretty cool.
12
SPEAKER_02
You you could actually go to a comedy club They had a stage set up like wow, it s kind of crazy
12
SPEAKER_02
But the gap between that and accepting it as real is pretty far.
12
SPEAKER_02
But that could be bridged with technology really quickly.
12
SPEAKER_02
Haptic feedback and especially some sort of a neural interface, whether it s Neuralink or something that you wear like that Google one where the guy was wearing it and he was asking questions and he was getting the answers fed through his head so he got answers to any questions.
12
SPEAKER_02
When that comes about, when you re getting sensory input, and then you re having real-life interactions with people, as that scales up exponentially, it s going to be indiscernible, which is the whole simulation hypothesis.
13
SPEAKER_00
another way that could happen that is maybe even less dependent on directly plugging into like human brains and all that sort of thing, which is, so every time, we don t know, and this is super speculative, I m just gonna carve this out as Jeremy s being super like, guesswork here, nobody knows.
13
SPEAKER_00
Go for it, Jeremy.
13
SPEAKER_00
Giddy up.
13
SPEAKER_00
So, you ve got this idea that every time you have a model that generates an output,
13
SPEAKER_00
it s having to kind of tap into a model, a kind of mental image, if you will, of the way the world is.
13
SPEAKER_00
It kind of, in a sense, you could argue, instantiates maybe a simulation of how the world is.
13
SPEAKER_00
In other words, to take it to the extreme, not saying this is what s actually going on, in fact, I would even say this is probably, sorry, this is certainly not what s going on with current models, but eventually maybe, who knows,
13
SPEAKER_00
Every time you generate the next word in the token prediction, you re having to load up this entire simulation maybe of all the data that the model has ingested, which could basically include all of known physics at a certain point.
13
SPEAKER_00
I mean, again, super speculative, but it s literally every token that the chatbot predicts could be associated with a standup of an entire simulated environment.
13
SPEAKER_00
Who knows?
13
SPEAKER_00
Not saying this is the case, but just like when you think about what is the mechanism that would produce the most simulated worlds as fast?
13
SPEAKER_02
Accurate.
13
SPEAKER_02
Also the most accurate prediction.
13
SPEAKER_02
Like if you fully simulate, you know, a world that s potentially going to give you very accurate predictions.
13
SPEAKER_00
Like, it s possible, but it kind of speaks to that question of consciousness, too.
13
SPEAKER_00
Right, what is it?
13
SPEAKER_00
Yeah, no idea.
13
SPEAKER_01
Yeah, we re very cocky about that.
13
SPEAKER_00
Yeah.
13
SPEAKER_01
Yeah, I mean, and there s emerging evidence that plants are not just consciousness, they actually communicate, which is real weird.
13
SPEAKER_01
Because, like, then what is that?
13
SPEAKER_01
If it s not in the neurons, if it s not in the brain, and it exists in everything, does it exist in soil?
13
SPEAKER_01
Is it in trees?
13
SPEAKER_01
What is a butterfly thinking?
13
SPEAKER_01
Actors have a limited capacity to express itself.
13
SPEAKER_01
We re so ignorant.
13
SPEAKER_01
Yeah, but we re also very arrogant, you know, because we re the shit.
13
SPEAKER_01
We re people.
13
SPEAKER_02
Yeah, bingo.
13
SPEAKER_01
Which allows us to have the hubris to make something like AI.
13
SPEAKER_02
Yeah, and the worst episodes in the history of our species are, I think like Jeremy said, have been when we looked at others as though they were not people and treated them that way.
13
SPEAKER_00
And you can kind of see how, so I don t know, there s, when you look at like what humans think is conscious and what humans think is not conscious,
13
SPEAKER_00
There s a lot of human chauvinism, I guess you d call it, that goes into that.
13
SPEAKER_00
We look at a dog, we re like, oh, it must be conscious because it licks me.
13
SPEAKER_00
It acts as if it loves me.
13
SPEAKER_00
There are all these outward indicators of a mind there.
13
SPEAKER_00
But when you look at cells, cells communicate with their environments in ways that are completely different and alien to us.
13
SPEAKER_00
You know there are inputs and outputs and all that kind of thing you can also look at the higher scale the human superorganism We talked about all those human beings interacting together to form this like you know Planet wide organism, what is that thing conscious is there some kind of consciousness?
13
SPEAKER_01
We could describe and then what the fuck is spooky action at a distance.
13
SPEAKER_01
You know what s going on in the quantum You know when you get to that.
13
SPEAKER_01
It s like okay.
13
SPEAKER_01
What are you saying?
13
SPEAKER_01
Like these things are expressing information faster than the speed of light?
13
SPEAKER_01
What?
13
SPEAKER_01
Dude, you re trying to trigger my quantum fuzzies here.
13
SPEAKER_00
Please.
13
SPEAKER_02
This guy did grad school in quantum mechanics.
13
SPEAKER_00
Oh, please.
13
SPEAKER_00
I m really sorry.
13
SPEAKER_00
Well, how bonkers is it?
13
SPEAKER_00
Oh, it s like a seven, Joe.
13
SPEAKER_00
It s magic.
13
SPEAKER_00
It s like a seven, yeah.
13
SPEAKER_00
It s very bonkers.
13
SPEAKER_00
So, okay.
13
SPEAKER_00
One of the problems right now with physics is that we have
13
SPEAKER_00
So imagine all the experimental data that we ve ever collected, all the Bunsen burner experiments and all the ramps and cars sliding down inclines, whatever, that s all a body of data.
13
SPEAKER_00
To that data, we re gonna fit some theories, right?
13
SPEAKER_00
So we re gonna fit, basically, Newtonian physics is a theory that we try to fit to that data to try to explain it.
13
SPEAKER_00
Newtonian physics breaks because it doesn t account for a lot of those observations, a lot of those data points.
13
SPEAKER_00
Quantum physics is a lot better, but there s some weird areas where it still doesn t quite fit the bill, but it covers an awful lot of those data points.
13
SPEAKER_00
There s like a million different ways to tell the story of what quantum physics means about the world that are all mutually inconsistent.
13
SPEAKER_00
Like these are the different interpretations of the theory.
13
SPEAKER_00
Some of them say that, yeah, they re parallel universes.
13
SPEAKER_00
Some of them say that human consciousness is central to physics.
13
SPEAKER_00
Some of them say that like the future is predetermined from the past.
13
SPEAKER_00
And all of those theories fit perfectly to all the points that we have so far.
13
SPEAKER_00
But they tell a completely different story about what s true and what s not.
13
SPEAKER_00
And some of them even have something to say about, for example, consciousness.
13
SPEAKER_00
And so, in a weird way, the fact that we haven t cracked the nut on any of that stuff means we really have no shot at understanding the consciousness equation, the sentience equation when it comes to AI or whatever else.
13
SPEAKER_02
But for action at a distance, one of the spooky things about that is that you can t actually get it to communicate anything concrete at a distance.
13
SPEAKER_02
Everything about the laws of physics conspires
13
SPEAKER_02
to stop you from communicating faster than light, including what s called action at a distance.
13
SPEAKER_00
As far as we currently know.
13
SPEAKER_02
As far as we know.
13
SPEAKER_00
And that s the problem.
13
SPEAKER_00
So if you look at the leap from Newtonian physics to Einstein, with Newton, we re able to explain a whole bunch of shit.
13
SPEAKER_00
The world seems really simple.
13
SPEAKER_00
It s forces and it s masses, and that s basically it.
13
SPEAKER_00
You got objects.
13
SPEAKER_00
But then people go, oh, look at the orbit of Mercury.
13
SPEAKER_00
It s a little wobbly.
13
SPEAKER_00
We gotta fix that.
13
SPEAKER_00
And it turns out that if you re gonna fix that one stupid wobbly orbit, you need to completely change your whole picture of what s true in the world.
13
SPEAKER_00
All of a sudden, you ve got a world where space and time are linked together, you have to, they get bent by gravity, they get bent by energy, there s all kinds of weird shit that happens with time and length control, like all that stuff, all just to account for this one stupid observation of the wobbly orbit of frickin  Mercury.
13
SPEAKER_00
And the challenge is this might actually end up being true with quantum mechanics.
13
SPEAKER_00
In fact, we know quantum mechanics is broken because it doesn t actually fit with our theory of general relativity from Einstein.
13
SPEAKER_00
We can t make them kind of play nice with each other at certain scales.
13
SPEAKER_00
And so there s our wobbly orbit.
13
SPEAKER_00
So now if we re going to solve that problem, if we re going to create a unified theory, we re going to have to step outside of that.
13
SPEAKER_00
almost certainly or it seems very likely we ll have to refactor our whole picture of the universe in a way that s just as fundamental as the leap from Newton to Einstein.
13
SPEAKER_00
This is where Scarlett Johansson comes in.
13
SPEAKER_01
I was just I can do this.
13
SPEAKER_00
You don t have to do this.
13
SPEAKER_01
I can take this off your hands.
13
SPEAKER_01
Let me solve all of this.
13
SPEAKER_01
Really complicated.
13
SPEAKER_01
But because you have a simian brain, you have a little monkey brain that s just like super advanced, but it s really shitty.
13
SPEAKER_01
You know what?
13
SPEAKER_01
That s harsh.
13
SPEAKER_01
But it sounded really hot.
13
SPEAKER_01
Yeah, especially if you have the horse, Scarlett Johansson, from her, like the bedtime voice.
13
SPEAKER_02
So you re the one that they got to do the voice of Skye.
13
SPEAKER_02
Yes, it s me.
13
SPEAKER_02
That was you.
13
SPEAKER_02
Oh, dude, I did my girl voice.
13
SPEAKER_00
On the sexiness of Scarlett Johansson s voice, so Opening Eye at one point said, I can t remember if it was Sam or Opening Eye itself, they were like, hey, so the one thing we re not gonna do is optimize for engagement.
13
SPEAKER_00
with our products.
13
SPEAKER_00
And when I first heard the sexy, sultry, seductive Scarlett Johansson voice, and I finished cleaning up my pants, I was like, damn, that seems like optimization for something.
13
SPEAKER_00
I don t know if it s like... Right.
13
SPEAKER_01
Otherwise you get Richard Simmons to do the voice.
13
SPEAKER_01
Exactly.
13
SPEAKER_01
That s my whole thing.
13
SPEAKER_02
If you want to turn people on, there s a lot of other options.
13
SPEAKER_02
That s like that s that s an optimization for like growth of Google s thing.
13
SPEAKER_01
Yeah Yeah, Google s got to do Richard Simmons.
13
SPEAKER_01
Yeah, what are they gonna do?
13
SPEAKER_01
Um boy so
13
SPEAKER_01
Do you think that AI, if it does get to an AGI place, could it possibly be used to solve some of these puzzles that have eluded our simple minds totally?
13
SPEAKER_01
Totally, I mean, even before.
13
SPEAKER_02
So the potential advancements.
13
SPEAKER_01
Even before AGI.
13
SPEAKER_01
So we can t all of them just.
13
SPEAKER_02
No, it s like, it s so potentially positive, and even before AGI.
13
SPEAKER_02
Because remember, we talked about how these systems make mistakes that are totally different from the kinds of mistakes we make, right?
13
SPEAKER_02
And so what that means is, we make a whole bunch of mistakes that an AI would not make, especially as it gets closer to our capabilities.
13
SPEAKER_02
I was reading this thought by Kevin Scott, who s the CTO of Microsoft.
14
SPEAKER_02
with a number of people that, you know, in the next few years, an AI is going to solve this particular mathematical theorem, conjecture, called the Riemann hypothesis.
14
SPEAKER_02
It s like, you know, how spaced out are the prime numbers, whatever, some like mathematical thing that for a hundred years plus people have just like scratched their heads over.
14
SPEAKER_02
these things are incredibly valuable.
14
SPEAKER_02
His expectation is it s not going to be an AGI, it s going to be a collaboration between a human and an AI.
14
SPEAKER_02
Even on the way to that, before you hit AGI, there s a ton of value to be had because these systems think so fast.
14
SPEAKER_02
They re tireless compared to us.
14
SPEAKER_02
They have different view of the world and can solve problems potentially in interesting ways.
14
SPEAKER_02
So
14
SPEAKER_02
Yeah, there s tons and tons of positive value there.
14
SPEAKER_01
So AlphaFold2 was the first, so let me take a step back, there s this really critical problem in molecular biology where you have, so proteins, which are just a, it s a sequence of building blocks.
14
SPEAKER_01
The building blocks are called amino acids.
14
SPEAKER_01
And each of the amino acids, they have different structures.
14
SPEAKER_01
And so once you finish stringing them together, they ll naturally kind of fold together in some interesting shape.
14
SPEAKER_01
And that shape gives that overall protein its function.
14
SPEAKER_01
So if you can predict the shape, the structure of a protein based on its amino acid sequence, you can start to do shit like design new drugs, you can solve all kinds of problems.
14
SPEAKER_01
This is the expensive crown jewel problem of the field.
14
SPEAKER_01
AlphaFold2 in one swoop was like, oh, we can solve this problem basically much better than a lot of even empirical methods.
14
SPEAKER_01
Now AlphaFold3 comes out, they re like, yeah, and now we can do it if we tack on a bunch of, yeah, there it is.
14
SPEAKER_01
If we can tack on a bunch.
14
SPEAKER_00
Look at this quote.
14
SPEAKER_00
AlphaFold3 predicts the structure and interactions of all of life s molecules.
14
SPEAKER_00
What in the fuck, kids?
14
SPEAKER_00
Introducing AlphaFold3, a new AI model developed by Google DeepMind and Isomorphic Labs.
14
SPEAKER_00
By accurately predicting the structure of proteins, DNA, RNA, ligands and more, and how they interact, we hope it will transform our understanding of the biological world and drug discovery.
14
SPEAKER_01
So this is like just your typical Wednesday in the world of AI, right?
14
SPEAKER_01
Because it s happening so quickly.
14
SPEAKER_02
Yeah, that s it.
14
SPEAKER_02
So it s like, oh yeah, another revolution happened this month.
14
SPEAKER_00
And it s all happening so fast and our timeline is so flooded with data that everyone s kind of unaware of the pace of it all, but it s happening at such a strange exponential rate.
14
SPEAKER_01
For better and for worse, right?
14
SPEAKER_01
And this is definitely on the better side of the equation.
14
SPEAKER_01
There s a bunch of stuff like, one of the papers that actually Google DeepMind came out with earlier in the year was in a single advance, like a single paper, a single AI model they built,
14
SPEAKER_01
They expanded the set of stable materials.
14
SPEAKER_00
Coffee s terrible.
14
SPEAKER_00
I ll just tell you right now.
14
SPEAKER_00
Jamie, it sucks.
14
SPEAKER_00
Poor Jamie.
14
SPEAKER_00
I love terrible coffee.
14
SPEAKER_00
The water never got hot.
14
SPEAKER_00
Yeah, that s what it is.
14
SPEAKER_00
It s just never never really brewed.
14
SPEAKER_00
It s terrible terrible coffee is my favorite Yeah, I can solve that problem to probably wait you try this terrible for coffee though.
14
SPEAKER_00
You like this fucking terrible He looks like terrible mmm terrible yeah, I could just see that calculation of like like if you re dating a really hot girl She cooks for you like thank you.
14
SPEAKER_00
This is amazing.
14
SPEAKER_00
This is the best macaroni and cheese ever it s
14
SPEAKER_01
If in fairness if Scarlett Johansson s voice was actually giving you that I believe is the best car Keep talking Yeah, so so there s this one this one paper that came out and they re like hey by the way we ve increased the set of stable materials known to humanity by a factor of 10 and
14
SPEAKER_01
So, like, if on Monday we knew about, you know, 100,000 stable materials, we now know about a million.
14
SPEAKER_01
They were then validated, replicated by Berkeley University or a bunch of them as a proof of concept.
14
SPEAKER_02
And this is from, like, you know, the stable materials we knew before, like that Wednesday, were from ancient times, like the ancient Greeks, like, discovered some shit, the Romans discovered some shit, the Middle Ages discovered some shit, and then it s like, oh yeah, yeah, all that, that was really cute.
14
SPEAKER_02
Like, boom.
14
SPEAKER_02
Instantly, one step.
14
SPEAKER_01
And that s amazing.
14
SPEAKER_00
We should be celebrating that.
14
SPEAKER_00
We re going to have great phones in 10 years.
14
SPEAKER_01
Dude, we ll be able to get addicted to feeds that we haven t even thought of.
14
SPEAKER_00
So, I mean, you re making me feel a little more positive.
14
SPEAKER_00
Overall, there s going to be so many beneficial aspects to AI.
14
SPEAKER_00
Oh yeah.
14
SPEAKER_00
And it s just, what it is, is just...
14
SPEAKER_00
Unbelievably transformative event that we re living through.
14
SPEAKER_02
Yeah, it s power and power can be good and it can be bad and that doesn t Yeah, an immense power.
14
SPEAKER_02
It can be immensely good or immensely bad
14
SPEAKER_01
And we re just in this, who knows?
14
SPEAKER_01
We just need to structurally set ourselves up so that we can reap the benefits and mind the downside risk.
14
SPEAKER_01
That s what it s always about, but the regulatory story has to unfold that way.
14
SPEAKER_00
Well, I m really glad that you guys have the ethics to get out ahead of this and to talk about it with so many people and to really blare this message out.
14
SPEAKER_00
Because I don t think ... There s a lot of people that ... I had Marc Andreessen on, who s brilliant, but he s like, all in.
14
SPEAKER_00
It s going to be great.
14
SPEAKER_00
And maybe he s right.
14
SPEAKER_00
Maybe he s right.
14
SPEAKER_02
Yeah, but I mean you have to hear all the different perspectives and I mean like Massive massive props honestly go out to the team at the State Department that we worked with one of the things also is over the course of the investigation the way it was structured was
14
SPEAKER_02
It wasn t like a contract, and they farmed it out, and we went out.
14
SPEAKER_02
It was the two teams actually worked together.
14
SPEAKER_02
The two teams together, the State Department and us, we went to London, UK.
14
SPEAKER_02
We talked and sat down with DeepMind.
14
SPEAKER_02
We went to San Francisco.
14
SPEAKER_02
sat down with Sam Altman and his policy team, we sat down with Anthropic, all of us together.
14
SPEAKER_02
One of the major reasons why we were able to publish so much of the whistleblower stuff is that those very individuals were in the rooms with us when we found out this shit and they were like,
14
SPEAKER_02
Oh, fuck, like the world needs to know about this.
14
SPEAKER_02
And so they were pushing internally for a lot of this stuff to come out that otherwise would not.
14
SPEAKER_02
And I also got to say, like, I just want to memorialize this to that investigation.
14
SPEAKER_02
When we went around the world, we were working with some of the most elite people in the government that I didn t I would not have guessed existed.
14
SPEAKER_02
That was honestly more.
14
SPEAKER_02
Well, I can be.
14
SPEAKER_02
It s hard to be specific.
14
SPEAKER_00
Did you see any UFOs?
14
SPEAKER_00
Tell me the truth.
14
SPEAKER_01
Did they take you to the hangar?
14
SPEAKER_01
No, no.
14
SPEAKER_01
There s no hangar.
14
SPEAKER_00
Yeah, I believe you.
14
SPEAKER_00
Can we cut that?
14
SPEAKER_00
Can we cut that?
14
SPEAKER_01
There s no hangar, Joe.
14
SPEAKER_01
Don t worry, sweetie.
14
SPEAKER_01
Don t worry.
14
SPEAKER_02
We didn t we didn t go that far down the rabbit hole I m a you know, we went pretty far down the rabbit hole and yeah, there are individuals who are just absolutely absolutely elite like the level of capability the amount that our teams gelled together at certain points is
14
SPEAKER_02
the stakes, like the stuff we did, the stuff they made happen for us in terms of bringing us, they brought together like a hundred folks from across the government to discuss like AI on the path to AGI and go through the recommendations that we had.
14
SPEAKER_01
Oh yeah, this was pretty cool actually.
14
SPEAKER_01
It was like the first, basically the first time the US government came together and seriously looked at the prospect of AGI and the risks there.
14
SPEAKER_01
And we had, it was wild.
14
SPEAKER_01
I mean, again, it s like- That was in November.
14
SPEAKER_01
It s us two frigging Yahoos, like what the hell do we know, and our amazing team.
14
SPEAKER_01
And it was, yeah, referred to by, there was a senior White House rep there who was like, yeah, this is a watershed moment in U.S.
14
SPEAKER_01
history.
14
SPEAKER_00
Wow.
14
SPEAKER_00
Well, that s encouraging.
14
SPEAKER_00
Because, again, people do like to look at the government as the DMV.
14
SPEAKER_00
Yeah.
14
SPEAKER_00
Or the worst aspects of bureaucracy.
14
SPEAKER_01
There s missing room for things like congressional hearings on these whistleblower events.
14
SPEAKER_01
Certainly congressional hearings that we talked about on the idea of liability and licensing and what regulatory agencies we need just to start to get to the meat on the bone on this issue.
14
SPEAKER_01
Opening this up, I think, is really important.
14
SPEAKER_00
Well, shout out to the part of the government that s good.
14
SPEAKER_00
Shout out to the government that gets it, that s competent and awesome.
14
SPEAKER_00
And shout out to you guys, because it s heady stuff.
14
SPEAKER_00
It s very difficult to grasp.
14
SPEAKER_00
Even in having this conversation with you, I still don t know how to feel about it.
14
SPEAKER_00
I think I m at least slightly optimistic that the potential benefits are going to be huge.
14
SPEAKER_00
But what a weird passage we re about to enter into.
14
SPEAKER_00
It s the unknown.
14
SPEAKER_00
Yeah, truly.
14
SPEAKER_00
Thank you, gentlemen.
14
SPEAKER_00
Really appreciate your time.
14
SPEAKER_00
Appreciate what you re doing.
14
SPEAKER_00
Thank you.
14
SPEAKER_00
People want to know more.
14
SPEAKER_00
Where should they go?
14
SPEAKER_00
What should they follow?
15
SPEAKER_04
I guess Gladstone.ai slash action plan is one that has our action plan.
15
SPEAKER_00
Gladstone.ai, all our stuff is there.
15
SPEAKER_04
I should mention too, I have this little podcast called Last Week in AI.
15
SPEAKER_04
We cover sort of last week s events and it s all about the sort of lens of- You don t have to do that every hour.
15
SPEAKER_01
Yeah.
15
SPEAKER_01
Last hour in AI.
15
SPEAKER_01
It s like a week is not enough time.
15
SPEAKER_01
We could be at war.
15
SPEAKER_03
Our list of stories keeps getting longer.
15
SPEAKER_02
Aliens landing.
15
SPEAKER_02
Yeah, anything can happen.
15
SPEAKER_02
Time travel.
15
SPEAKER_02
You ll hear it there first.
15
SPEAKER_02
Yeah.
15
SPEAKER_02
All right.
15
SPEAKER_02
Well, thank you guys.
15
SPEAKER_02
Thank you very much.
15
SPEAKER_02
Appreciate it.
15
SPEAKER_02
Bye, everybody.