0
SPEAKER_01
The following is a conversation with Chris Urmson.
0
SPEAKER_01
He was the CTO of the Google self-driving car team, a key engineer and leader behind the Carnegie Mellon University autonomous vehicle entries in the DARPA Grand Challenges and the winner of the DARPA Urban Challenge.
0
SPEAKER_01
Today, he s the CEO of Aurora Innovation, an autonomous vehicle software company he started with Sterling Anderson, who was the former director of Tesla Autopilot, and Drew Bagnell, Uber s former autonomy and perception lead.
0
SPEAKER_01
Chris is one of the top roboticists and autonomous vehicle experts in the world, and a longtime voice of reason in a space that is shrouded in both mystery and hype.
0
SPEAKER_01
He both acknowledges the incredible challenges involved in solving the problem of autonomous driving and is working hard to solve it.
0
SPEAKER_01
This is the Artificial Intelligence Podcast.
0
SPEAKER_01
If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support it on Patreon, or simply connect with me on Twitter at LexFriedman, spelled F-R-I-D-M-A-N.
0
SPEAKER_01
And now, here s my conversation with Chris Armisen.
0
SPEAKER_01
You were part of both the DARPA Grand Challenge and the DARPA Urban Challenge teams at CMU with Red Whitaker.
0
SPEAKER_01
What technical or philosophical things have you learned from these races?
0
SPEAKER_00
I think the high order bit was that it could be done.
0
SPEAKER_00
I think that was the thing that was incredible about the first, the Grand Challenges.
0
SPEAKER_00
that I remember you know I was a grad student at Carnegie Mellon and there we it was kind of this dichotomy of
0
SPEAKER_00
It seemed really hard, so that would be cool and interesting.
0
SPEAKER_00
But at the time, we were the only robotics institute around.
0
SPEAKER_00
And so if we went into it and fell on our faces, that would be embarrassing.
0
SPEAKER_00
So I think just having the will to go do it, to try to do this thing that at the time was marked as darn near impossible, and then after a couple of tries, be able to actually make it happen, I think that was
0
SPEAKER_00
And that was really exciting.
0
SPEAKER_01
But at which point did you believe it was possible?
0
SPEAKER_01
Did you, from the very beginning, did you personally, because you re one of the lead engineer, you actually had to do a lot of the work.
0
SPEAKER_00
Yeah, I was the technical director there and did a lot of the work along with a bunch of other really good people.
0
SPEAKER_00
Did I believe it could be done?
0
SPEAKER_00
Yeah, of course, right?
0
SPEAKER_00
Like, why would you go do something you thought was impossible, completely impossible?
0
SPEAKER_00
We thought it was going to be hard.
0
SPEAKER_00
We didn t know how we re going to be able to do it.
0
SPEAKER_00
We didn t know if we d be able to do it the first time.
0
SPEAKER_00
Turns out we couldn t.
0
SPEAKER_00
that yeah I guess you have to I think there s a certain benefit to naivete right that if you don t know how hard something really is you you try different things and you know it gives you an opportunity that others who are you know wiser maybe don t don t have.
0
SPEAKER_01
What were the biggest pain points?
0
SPEAKER_01
Mechanical, sensors, hardware, software, algorithms for mapping, localization, just general perception control, hardware, software, first of all.
0
SPEAKER_00
I think that s the joy of this field, is that it s all hard.
0
SPEAKER_00
And that you have to be good at each part of it.
0
SPEAKER_00
So for the urban challenges, if I look back at it from today, it should be easy today.
0
SPEAKER_00
That it was a static world.
0
SPEAKER_00
There weren t other actors moving through it, is what that means.
0
SPEAKER_00
It was out in the desert, so you get really good GPS.
0
SPEAKER_00
So that went, and we could map it roughly.
0
SPEAKER_00
And so in retrospect now, it s within the realm of things we could do back then.
0
SPEAKER_00
Just actually getting the vehicle, and there s a bunch of engineering work to get the vehicle so that we could control it and drive it.
0
SPEAKER_00
That s still a pain today, but it was even more so back then.
0
SPEAKER_00
And then the uncertainty of exactly what they wanted us to do.
0
SPEAKER_00
was part of the challenge as well.
0
SPEAKER_01
Right, you didn t actually know the track heading in.
0
SPEAKER_01
You knew approximately, but you didn t actually know the route, the route that was going to be taken.
0
SPEAKER_00
That s right.
0
SPEAKER_00
We didn t know the route.
0
SPEAKER_00
We didn t even really, the way the rules had been described, you had to kind of guess.
0
SPEAKER_00
So if you think back to that challenge, the idea was that the government would give us, DARPA would give us a set of waypoints and kind of the width that you had to stay within between the line that went between each of those waypoints.
0
SPEAKER_00
And so the most devious thing they could have done is set a kilometer-wide corridor across a field of scrub brush and rocks and said, go figure it out.
0
SPEAKER_00
Fortunately, it turned into basically driving along a set of trails, which is much more relevant to the application they were looking for.
0
SPEAKER_00
But no, it was a hell of a thing back in the day.
0
SPEAKER_01
So the legend, Red, was kind of leading that effort in terms of just broadly speaking.
0
SPEAKER_01
So you re a leader now.
0
SPEAKER_01
What have you learned from Red about leadership?
0
SPEAKER_00
I think there s a couple of things.
0
SPEAKER_00
One is, you know,
0
SPEAKER_00
Go and try those really hard things.
0
SPEAKER_00
That s where there is an incredible opportunity.
0
SPEAKER_00
I think the other big one, though, is to see people for who they can be, not who they are.
0
SPEAKER_00
It s one of the things that I actually, one of the deepest lessons I learned from Red was that he would look at undergraduates or graduate students
0
SPEAKER_00
and empower them to be leaders, to have responsibility, to do great things, that I think another person might look at them and think, oh, well, that s just an undergraduate student.
0
SPEAKER_00
What could they know?
0
SPEAKER_00
And so I think that kind of trust, but verify, have confidence in what people can become, I think, is a really powerful thing.
0
SPEAKER_01
So through that, let s just fast forward through the history.
0
SPEAKER_01
Can you maybe talk through the technical evolution of autonomous vehicle systems from the first two Grand Challenges to the Urban Challenge to today?
0
SPEAKER_01
Are there major shifts in your mind, or is it the same kind of technology, just made more robust?
0
SPEAKER_00
I think there s been some big, big steps.
0
SPEAKER_00
So for the Grand Challenge, the real
0
SPEAKER_00
technology that unlocked that was HD mapping.
0
SPEAKER_00
Prior to that a lot of the off-road robotics work had been done without any real prior model of what the vehicle was going to encounter.
0
SPEAKER_00
And so that innovation that the fact that we could get you know decimeter resolution
0
SPEAKER_00
models was really a big deal.
0
SPEAKER_00
And that allowed us to to kind of bound the complexity of the driving problem the vehicle had and allowed it to operate at speed because we could assume things about the environment that it was going to encounter.
0
SPEAKER_00
So that was a that was one of the that was the big step there.
0
SPEAKER_00
For the urban challenge,
0
SPEAKER_00
You know, one of the big technological innovations there was the multi beam LIDAR and be able to generate, um, high resolution, you know, mid to long range 3d models, the world, and use that for, you know, for understanding the world around the vehicle.
0
SPEAKER_00
And that was really a, you know, kind of a game changing technology.
0
SPEAKER_00
In parallel with that, we saw a bunch of other technologies that had been kind of converging half their day in the sun.
0
SPEAKER_00
So Bayesian estimation had been, you know, SLAM had been a big field
0
SPEAKER_00
in robotics, you would go to a conference a couple of years before that and every paper would effectively have SLAM somewhere in it.
0
SPEAKER_00
And so seeing those Bayesian estimation techniques play out on a very visible stage, I thought that was pretty exciting to see.
0
SPEAKER_01
And mostly SLAM was done based on LIDAR at that time.
0
SPEAKER_00
Well, yeah.
0
SPEAKER_00
And in fact, we weren t really doing SLAM per se, you know, in real time because we had a model ahead of time, we had a roadmap, but we were doing localization.
0
SPEAKER_00
And we re using, you know, the LIDAR or the cameras, depending on, you know, who exactly was doing it to localize to a model of the world.
0
SPEAKER_00
And I thought that was
0
SPEAKER_00
That was a big step from kind of naively trusting GPS, INS before that.
0
SPEAKER_00
And again, like lots of work had been going on in this field.
0
SPEAKER_00
Certainly this was not doing anything particularly innovative in SLAM or in localization, but it was seeing that technology necessary in a real application on a big stage, I thought was very cool.
0
SPEAKER_01
So for the urban challenge, those are already maps constructed offline.
0
SPEAKER_00
Yes.
0
SPEAKER_01
In general.
0
SPEAKER_01
Okay.
0
SPEAKER_01
And, uh, did people do that individually, individual teams do it individually?
0
SPEAKER_01
Uh, so they had their own different, different approaches there, or did everybody kind of share that information at least intuitively?
0
SPEAKER_00
So DARPA gave all the teams a model of the world, a map.
0
SPEAKER_00
And then one of the things that we had to figure out back then was, and it s still one of these things that trips people up today, is actually the coordinate system.
0
SPEAKER_00
uh so you get a latitude longitude and you know to so many decimal places you don t really care about kind of the ellipsoid of the earth that s being used uh but when you want to get to 10 centimeter or centimeter resolution uh you care whether the the coordinate system is you know nads 83 or wgs 84 or
0
SPEAKER_00
You know, these are different ways to describe both the kind of non-sphericalness of the Earth, but also kind of the, I think, I can t remember which one, the tectonic shifts that are happening and how to transform, you know, the global datum as a function of that.
0
SPEAKER_00
So, you know, getting a map and then actually matching it to reality to centimeter resolution, that was kind of interesting and fun back then.
0
SPEAKER_01
So how much work was the perception doing there?
0
SPEAKER_01
So how much were you relying on localization based on maps without using perception to register to the maps?
0
SPEAKER_01
I guess the question is how advanced was perception at that point?
0
SPEAKER_00
It s certainly behind where we are today, right?
0
SPEAKER_00
We re more than a decade since the urban challenge, but the core of it was there.
0
SPEAKER_00
That we were tracking vehicles.
0
SPEAKER_00
We had to do that at 100 plus meter range because we had to merge with other traffic.
0
SPEAKER_00
We were using, you know, Bayesian, again, Bayesian estimates for state of these vehicles.
0
SPEAKER_00
We had to deal with a bunch of the problems that you think of today, of predicting where that vehicle s going to be a few seconds into the future.
0
SPEAKER_00
We had to deal with the fact that there were multiple hypotheses for that, because a vehicle at an intersection might be going right, or it might be going straight, or it might be making a left turn.
0
SPEAKER_00
And we had to deal with the challenge of the fact that our behavior was going to impact the behavior of that other operator.
0
SPEAKER_00
You know, we did a lot of that in relatively naive ways, but it kind of worked.
0
SPEAKER_01
Still had to have some kind of solution.
0
SPEAKER_00
Yeah.
0
SPEAKER_01
And so where does that, 10 years later, where does that take us today from that artificial city construction to real cities to the urban environment?
0
SPEAKER_00
Yeah.
0
SPEAKER_00
I think the biggest thing is that the, you know, the actors are truly unpredictable.
0
SPEAKER_00
that most of the time, you know, the drivers on the road, the other road users are out there behaving well, but every once in a while they re not.
0
SPEAKER_00
The variety of other vehicles is, you know, you have all of them.
0
SPEAKER_01
In terms of behavior, in terms of perception, or both?
0
SPEAKER_01
Both.
0
SPEAKER_00
That we have, you know, back then we didn t have to deal with cyclists, we didn t have to deal with pedestrians, didn t have to deal with traffic lights.
0
SPEAKER_00
you know, the scale over which that you have to operate is now, you know, is much larger than, you know, the airbase that we were thinking about back then.
0
SPEAKER_01
So what, uh, easy question.
0
SPEAKER_01
What do you think is the hardest part about driving?
0
SPEAKER_01
Easy question.
0
SPEAKER_01
Yeah.
0
SPEAKER_01
Uh, no, I m, I m, I m joking.
0
SPEAKER_01
I, I m, I m sure no, nothing really jumps out at you as one thing, but, uh, in, in the jump from the urban challenge to the real world, is there something that s a particular, you foresee as very serious, difficult challenge?
0
SPEAKER_00
I think the, the most fundamental difference is,
0
SPEAKER_00
that were doing it for real and that in that environment it was both a limited complexity environment because certain actors weren t there because you know the roads were maintained there were barriers keeping people separate from from robots at the time
0
SPEAKER_00
And it only had to work for 60 miles, which looking at it from 2006, it had to work for 60 miles.
0
SPEAKER_00
Looking at it from now, we want things that will go and drive for half a million miles.
0
SPEAKER_00
And it s just a different game.
0
SPEAKER_01
So how important, you said LiDAR came into the game early on, and it s really the primary driver of autonomous vehicles today as a sensor.
0
SPEAKER_01
So how important is the role of LiDAR in the sensor suite in the near term?
0
SPEAKER_00
So I think it s essential.
0
SPEAKER_00
You know, I believe, but I also believe the cameras are essential and I believe the radar is essential.
0
SPEAKER_00
I think that you really need to use a composition of data from these different sensors if you want the thing to really be robust.
0
SPEAKER_01
The question I want to ask, let s see if we can untangle it, is what are your thoughts on the Elon Musk provocative statement that LiDAR is a crutch?
0
SPEAKER_01
that is a kind of, I guess, growing pains.
0
SPEAKER_01
And that much of the perception task can be done with cameras.
0
SPEAKER_00
So I think it is undeniable that people walk around without lasers in their foreheads.
0
SPEAKER_00
And they can get into vehicles and drive them.
0
SPEAKER_00
And so there s an existence proof that you can drive using passive vision.
0
SPEAKER_00
no doubt can t argue with that in terms of sensors yeah so yeah in terms of sensors right so like there s there s an example that you know we all go do it many of us every day in terms of uh lidar being a crutch sure but but you know in the same way that uh you know the combustion engine was a crutch on the path to an electric vehicle in the same way that you know any technology ultimately gets
0
SPEAKER_00
Replaced by some superior technology in the future.
0
SPEAKER_00
And really what the way that I look at this is
0
SPEAKER_00
that the way we get around on the ground the way that we use transportation is broken um and that we have you know this this you know what was i think the number i saw this morning 37 000 americans killed uh last year on our roads and that s just not acceptable and so tech any technology that we can bring to bear that accelerates the this techno you know self-driving technology coming to market and saving lives is technology we should be using
0
SPEAKER_00
And it feels just arbitrary to say, well, I m not OK with using lasers, because that s whatever.
0
SPEAKER_00
But I am OK with using an 8-megapixel camera or a 16-megapixel camera.
0
SPEAKER_00
These are just bits of technology.
0
SPEAKER_00
And we should be taking the best technology from the tool bin that allows us to go and solve a problem.
0
SPEAKER_01
The question I often talk to, well, obviously you do as well, to automotive companies, and if there s one word that comes up more often than anything, it s cost, and trying to drive cost down.
0
SPEAKER_01
So while it s true that it s a tragic number, the $37,000,
0
SPEAKER_01
The question is, and I m not the one asking this question, because I hate this question, but we want to find the cheapest sensor suite that creates a safe vehicle.
0
SPEAKER_01
So in that uncomfortable trade-off, do you foresee LiDAR coming down in cost in the future, or do you see a day where level four autonomy is possible without LiDAR?
0
SPEAKER_00
I see both of those, but it s really a matter of time.
0
SPEAKER_00
And I think really, maybe I would talk to the question you asked about, you know, the cheapest sensor.
0
SPEAKER_00
I don t think that s actually what you want.
0
SPEAKER_00
What you want is a sensor suite that is economically viable.
0
SPEAKER_00
And then after that, everything is about margin and driving cost out of the system.
0
SPEAKER_00
What you also want is a sense suite that works.
0
SPEAKER_00
And so it s great to tell a story about how it would be better to have a self-driving system with a $50 sensor instead of a $500 sensor.
0
SPEAKER_00
But if the $500 sensor makes it work and the $50 sensor doesn t work, you know, who cares?
0
SPEAKER_00
So long as you can actually, uh, you know, have an economic offer, you know, there s an economic opportunity there.
0
SPEAKER_00
And the economic opportunity is important because that s how you actually have a sustainable business.
0
SPEAKER_00
And that s how you can actually see this come to scale and, and, and be out in the world.
0
SPEAKER_00
And so when I look at LIDAR, I see a technology that has
1
SPEAKER_00
no underlying fundamental expense to it.
1
SPEAKER_00
It s going to be more expensive than an imager because CMOS processes or FAB processes are dramatically more scalable than mechanical processes.
1
SPEAKER_00
But we still should be able to drive cost down substantially on that side.
1
SPEAKER_00
And then I also do think that with the right business model, you can absorb certainly more cost on the bill of materials.
1
SPEAKER_01
Yeah, if the sensor suite works, extra value is provided, thereby you don t need to drive costs down to zero.
1
SPEAKER_01
It s the basic economics.
1
SPEAKER_01
You ve talked about your intuition that level two autonomy is problematic because of the human factor of vigilance, decrement, complacency, overtrust, and so on, just us being human.
1
SPEAKER_01
We overtrust the system, we start doing even more so partaking in the secondary activities like smartphones and so on.
1
SPEAKER_01
Have your views evolved on this point in either direction?
1
SPEAKER_01
Can you speak to it?
1
SPEAKER_00
And I want to be really careful because sometimes this gets twisted in a way that I certainly didn t intend.
1
SPEAKER_00
Active safety systems are a really important technology that we should be pursuing and integrating into vehicles.
1
SPEAKER_00
And there s an opportunity in the near term to reduce accidents, reduce fatalities, and we should be pushing on that.
1
SPEAKER_00
Level two systems are systems where the vehicle is controlling two axes, so braking and throttle slash steering.
1
SPEAKER_00
And I think there are variants of level two systems that are supporting the driver that absolutely we should encourage to be out there.
1
SPEAKER_00
Where I think there s a real challenge is in the human factors part around this and the misconception from the public around the capability set that that enables and the trust they should have in it.
1
SPEAKER_00
And that is where I, you know, I kind of, I am actually incrementally more, you know, concerned around level three systems and, you know, how exactly a level two system is marketed and delivered.
1
SPEAKER_00
And, you know, how people, how much effort people have put into those human factors.
1
SPEAKER_00
I still believe several things around this.
1
SPEAKER_00
One is people will overtrust the technology.
1
SPEAKER_00
We ve seen over the last few weeks a spate of people sleeping in their Tesla.
1
SPEAKER_00
I watched an episode last night of Trevor Noah.
1
SPEAKER_00
talking about this and you know him you know this is a smart guy who s has a lot of resources at his disposal describing a tesla as a self-driving car and that why shouldn t people be sleeping in their tesla it s like well because it s not a self-driving car and it is not intended to be and you know these people will almost certainly you know die at some point
1
SPEAKER_00
or hurt other people.
1
SPEAKER_00
And so we need to really be thoughtful about how that technology is described and brought to market.
1
SPEAKER_00
I also think that because of the economic issue, economic challenges we were just talking about, that that technology path will, these level two driver assistance systems, that technology path will diverge from the technology path that we need to be on to actually deliver
1
SPEAKER_00
truly self-driving vehicles ones where you can get in it and sleep and have the Equivalent or better safety than you know a human driver behind the wheel Because the again the economics are very different In those two worlds and so that leads to you know divergent technology
1
SPEAKER_01
So you just don t see the economics of gradually increasing from level two and doing so quickly enough to where it doesn t cause safety, critical safety concerns.
1
SPEAKER_01
You believe that it needs to diverge at this point into different, basically different routes.
1
SPEAKER_00
And really that comes back to what are those L2 and L1 systems doing?
1
SPEAKER_00
And they are driver assistance functions.
1
SPEAKER_00
where the people that are marketing that responsibly are being very clear and putting human factors in place such that the driver is actually responsible for the vehicle and that the technology is there to support the driver.
1
SPEAKER_00
And the safety cases that are built around those are dependent on that driver attention and attentiveness.
1
SPEAKER_00
And at that point, you can kind of give up
1
SPEAKER_00
to some degree, for economic reasons, you can give up on, say, false negatives.
1
SPEAKER_00
And the way to think about this is for a foreclosure mitigation breaking system,
1
SPEAKER_00
If half the times the driver missed a vehicle in front of it, it hit the brakes and brought the vehicle to a stop, that would be an incredible, incredible advance in safety on our roads.
1
SPEAKER_00
That would be equivalent to seat belts.
1
SPEAKER_00
But it would mean that if that vehicle wasn t being monitored, it would hit one out of two cars.
1
SPEAKER_00
And so economically, that s a perfectly good solution for a driver assistance system.
1
SPEAKER_00
What you should do at that point, if you can get it to work 50% of the time, is drive the cost out of that so you can get it on as many vehicles as possible.
1
SPEAKER_00
But driving the cost out of it doesn t drive up performance on the false negative case.
1
SPEAKER_00
And so you ll continue to not have a technology that could really be available for a self-driven vehicle.
1
SPEAKER_01
So clearly the communication, and this probably applies to all four vehicles as well, the marketing and the communication of what the technology is actually capable of, how hard it is, how easy it is, all that kind of stuff is highly problematic.
1
SPEAKER_01
So say everybody in the world was perfectly communicated and were made to be completely aware of every single technology out there, what it s able to do.
1
SPEAKER_01
What s your intuition?
1
SPEAKER_01
And now we re maybe getting into philosophical ground.
1
SPEAKER_01
Is it possible to have a level two vehicle where we don t overtrust it?
1
SPEAKER_00
I don t think so.
1
SPEAKER_00
If people truly understood the risks and internalized it, then sure, you could do that safely.
1
SPEAKER_00
But that s a world that doesn t exist.
1
SPEAKER_00
If the facts are put in front of them, they re going to then combine that with their experience.
1
SPEAKER_00
And let s say they re using an L2 system, and they go up and down the 101 every day, and they do that for a month.
1
SPEAKER_00
And it just worked every day for a month.
1
SPEAKER_00
That s pretty compelling.
1
SPEAKER_00
At that point, just even if you know the statistics,
1
SPEAKER_00
You re like, well, I don t know.
1
SPEAKER_00
Maybe there s something funny about those.
1
SPEAKER_00
Maybe they re driving in difficult places.
1
SPEAKER_00
I ve seen it with my own eyes.
1
SPEAKER_00
It works.
1
SPEAKER_00
And the problem is that that sample size that they have, so it s 30 miles up and down, so 60 miles times 30 days, so 60, 180, 1,800 miles.
1
SPEAKER_00
that s a drop in the bucket compared to the, what, 85 million miles between fatalities.
1
SPEAKER_00
And so they don t really have a true estimate based on their personal experience of the real risks, but they re gonna trust it anyway because it s hard not to.
1
SPEAKER_00
It worked for a month.
1
SPEAKER_00
What s gonna change?
1
SPEAKER_01
So even if you start a perfect understanding of the system, your own experience will make it drift.
1
SPEAKER_01
I mean, that s a big concern.
1
SPEAKER_01
Over a year, over two years even.
1
SPEAKER_01
It doesn t have to be months.
1
SPEAKER_00
And I think that as this technology moves from what I would say is kind of the more technology savvy ownership group to, you know, the mass market.
1
SPEAKER_00
you may be able to have some of those folks who are really familiar with technology, they may be able to internalize it better.
1
SPEAKER_00
And your kind of immunization against this kind of false risk assessment might last longer.
1
SPEAKER_00
But as folks who aren t as savvy about that read the material and they compare that to their personal experience, I think there it s going to
1
SPEAKER_00
it s going to move more quickly.
1
SPEAKER_01
So your work, the program that you ve created at Google and now at Aurora is focused more on the second path of creating full autonomy.
1
SPEAKER_01
So it s such a fascinating
1
SPEAKER_01
I think it s one of the most interesting AI problems of the century, right?
1
SPEAKER_01
I just talked to a lot of people, just regular people.
1
SPEAKER_01
I don t know, my mom, about autonomous vehicles.
1
SPEAKER_01
And you begin to grapple with ideas of giving your life control over to a machine.
1
SPEAKER_01
It s philosophically interesting.
1
SPEAKER_01
It s practically interesting.
1
SPEAKER_01
So let s talk about safety.
1
SPEAKER_01
How do you think we demonstrate, you ve spoken about metrics in the past, how do you think we demonstrate to the world that an autonomous vehicle, an Aurora system is safe?
1
SPEAKER_00
this is one where it s difficult because there isn t a soundbite answer that we have to show a combination of uh work that was done diligently and thoughtfully and this is where something like a functional safety process is part of that is like here s here s the way we did the work
1
SPEAKER_00
that means that we were very thorough so you know if you believe that we what we said about this is the way we did it then you can have some confidence that we were thorough in in in the engineering work we put into the system uh and then on top of that the you know to kind of demonstrate that we weren t just thorough we were actually good at what we did
1
SPEAKER_00
there ll be a kind of a collection of evidence in terms of demonstrating that the capabilities work the way we thought they did, you know, statistically, and to whatever degree we can demonstrate that, both in some combination of simulation, some combination of unit testing and decomposition testing, and then some part of it will be on-road data.
1
SPEAKER_00
And I think the way we ll ultimately convey this to the public is there ll be clearly some conversation with the public about it, but we ll invoke the trusted nodes and that we ll spend more time being able to go into more depth with folks like NHTSA and other federal and state regulatory bodies.
1
SPEAKER_00
And given that they are operating in the public interest and they re trusted,
1
SPEAKER_00
that if we can show enough work to them that they re convinced, then I think we re in a pretty good place.
1
SPEAKER_01
That means you work with people that are essentially experts at safety to try to discuss and show.
1
SPEAKER_01
Do you think the answer is probably no, but just in case?
1
SPEAKER_01
Do you think there exists a metric?
1
SPEAKER_01
So currently people have been using number of disengagements and it quickly turns into a marketing scheme to just sort of, you alter the experiments you run to adjust.
1
SPEAKER_01
I think you ve spoken that you don t like.
1
SPEAKER_00
Don t love it.
1
SPEAKER_00
No.
1
SPEAKER_00
In fact, I was on the record telling DMV that I thought this was not a great metric.
1
SPEAKER_01
Do you think it s possible to create a metric, a number that, um, that could demonstrate safety outside of fatalities?
1
SPEAKER_00
So I do, and I think that it won t be just one number.
1
SPEAKER_00
So as we are internally grappling with this, and at some point we ll be able to talk more publicly about it, is how do we think about human performance in different tasks, say, detecting traffic lights or safely making a left turn across traffic?
1
SPEAKER_00
And what do we think the failure rates are for those different capabilities for people?
1
SPEAKER_00
And then demonstrating to ourselves and then ultimately folks in regulatory role and then ultimately the public that we have confidence that our system will work better than that.
1
SPEAKER_00
And so these individual metrics will kind of tell a compelling story ultimately.
1
SPEAKER_00
I do think at the end of the day, what we care about in terms of safety is lives saved and injuries reduced.
1
SPEAKER_00
And then ultimately, you know, kind of casualty dollars that people aren t having to pay to get their car fixed.
1
SPEAKER_00
And I do think that you can, you know, in aviation, they look at a kind of an event pyramid where, you know, a crash is at the top of that, and that s the worst event, obviously.
1
SPEAKER_00
And then there s injuries and, you know, near-miss events and whatnot, and, you know, violation of operating procedures.
1
SPEAKER_00
And you kind of build a statistical model of the relevance of the low-severity things to the high-severity things.
1
SPEAKER_00
And I think that s something we ll be able to look at as well.
1
SPEAKER_00
because an event per 85 million miles is statistically a difficult thing even at the scale of the U.S.
1
SPEAKER_00
to kind of compare directly.
1
SPEAKER_01
And that event, the fatality that s connected to an autonomous vehicle is
1
SPEAKER_01
significantly, at least currently, magnified in the amount of attention it gets.
1
SPEAKER_01
So that speaks to public perception.
1
SPEAKER_01
I think the most popular topic about autonomous vehicles in the public is the trolley problem formulation, right?
1
SPEAKER_01
Which has, let s not get into that too much, but is misguided in many ways.
1
SPEAKER_01
But it speaks to the fact that people are grappling with this idea of giving control over to a machine.
1
SPEAKER_01
So how do you win the hearts and minds of the people that autonomy is something that could be a part of their lives?
1
SPEAKER_00
I think you let them experience it.
1
SPEAKER_00
I think it s right.
1
SPEAKER_00
I think people should be skeptical.
1
SPEAKER_00
I think people should ask questions.
1
SPEAKER_00
I think they should doubt.
1
SPEAKER_00
Because this is something new and different.
1
SPEAKER_00
They haven t touched it yet.
1
SPEAKER_00
And I think it s perfectly reasonable.
1
SPEAKER_00
But at the same time, it s clear there s an opportunity to make the road safer.
1
SPEAKER_00
It s clear that we can improve access to mobility.
1
SPEAKER_00
It s clear that we can reduce the cost of mobility.
1
SPEAKER_00
And that once people try that and understand that it s safe and are able to use in their daily lives, I think it s one of these things that will just be obvious.
1
SPEAKER_00
And I ve seen this practically in demonstrations that I ve given where I ve had people come in and they re very skeptical and they get in a vehicle.
1
SPEAKER_00
My favorite one is taking somebody out on the freeway
1
SPEAKER_00
And we re on the 101 driving at 65 miles an hour.
1
SPEAKER_00
And after 10 minutes, they kind of turn and ask, is that all it does?
1
SPEAKER_00
And you re like, it s a self-driving car.
1
SPEAKER_00
I m not sure exactly what you thought it would do, right?
1
SPEAKER_00
But it becomes mundane.
1
SPEAKER_00
which is exactly what you want a technology like this to be, right?
1
SPEAKER_00
We don t really, when I turn the light switch on in here, I don t think about the complexity of those electrons being pushed down a wire from wherever it was and being generated.
1
SPEAKER_00
It s like I just get annoyed if it doesn t work, right?
1
SPEAKER_00
And what I value is the fact that I can do other things in this space.
1
SPEAKER_00
I can see my colleagues, I can read stuff on a paper, I can, you know,
1
SPEAKER_00
not be afraid of the dark.
1
SPEAKER_00
And I think that s what we want this technology to be like, is it s in the background and people get to have those life experiences and do so safely.
1
SPEAKER_01
So putting this technology in the hands of people speaks to scale of deployment.
1
SPEAKER_01
So what do you think
1
SPEAKER_01
The dreaded question about the future because nobody can predict the future.
1
SPEAKER_01
But just maybe speak poetically about when do you think we ll see a large scale deployment of autonomous vehicles, 10,000, those kinds of numbers.
1
SPEAKER_00
We ll see that within 10 years.
1
SPEAKER_00
I m pretty confident.
1
SPEAKER_01
What s an impressive scale?
1
SPEAKER_01
What moment?
1
SPEAKER_01
So you ve done the DARPA challenge where there s one vehicle.
1
SPEAKER_01
At which moment does it become, wow, this is serious scale?
1
SPEAKER_00
So I think the moment it gets serious is when we really do have a driverless vehicle operating on public roads and that we can do that kind of continuously.
1
SPEAKER_01
Without a safety driver?
1
SPEAKER_00
Without a safety driver in the vehicle.
1
SPEAKER_00
I think at that moment, we ve kind of crossed the zero to one threshold.
1
SPEAKER_00
And then it is about how do we continue to scale that?
1
SPEAKER_00
build the right business models, how do we build the right customer experience around it so that it is actually, you know, a useful product out in the world.
1
SPEAKER_00
And I think that is really, at that point, it moves from a, you know, what is this kind of mixed science engineering project into engineering and commercialization and really starting to deliver on the value that we all see here and, you know, and actually making that real in the world.
1
SPEAKER_01
What do you think that deployment looks like?
1
SPEAKER_01
Where do we first see the inkling of no safety driver, one or two cars here and there?
1
SPEAKER_01
Is it on the highway?
1
SPEAKER_01
Is it in specific routes in the urban environment?
1
SPEAKER_00
I think it s gonna be urban, suburban type environments.
1
SPEAKER_00
Yeah, with Aurora, when we thought about how to tackle this, it was kind of en vogue to think about trucking as opposed to urban driving.
1
SPEAKER_00
And again, the human intuition around this is that freeways are easier to drive on.
1
SPEAKER_00
Because everybody s kind of going in the same direction, and the lanes are a little wider, et cetera.
1
SPEAKER_00
And I think that that intuition is pretty good, except we don t really care about most of the time.
1
SPEAKER_00
We care about all of the time.
1
SPEAKER_00
And when you re driving on a freeway with a truck, say, at 70 miles an hour, and you ve got 70,000 pound load with you, that s just an incredible amount of kinetic energy.
1
SPEAKER_00
And so when that goes wrong, it goes really wrong.
1
SPEAKER_00
And that those challenges that you see occur more rarely, so you don t get to learn as quickly.
2
SPEAKER_00
They re incrementally more difficult than urban driving, but they re not easier than urban driving.
2
SPEAKER_00
And so I think this happens in moderate speed urban environments, because if two vehicles crash at 25 miles per hour, it s not good, but probably everybody walks away.
2
SPEAKER_00
And those events where there s the possibility for that occurring happen frequently.
2
SPEAKER_00
So we get to learn more rapidly.
2
SPEAKER_00
We get to do that with lower risk for everyone.
2
SPEAKER_00
And then we can deliver value to people that need to get from one place to another.
2
SPEAKER_00
And then once we ve got that solved, then the freeway driving part of this just falls out.
2
SPEAKER_00
But we re able to learn more safely, more quickly in the urban environment.
2
SPEAKER_01
So 10 years and then scale 20, 30 years.
2
SPEAKER_01
I mean, who knows if a sufficiently compelling experience is created.
2
SPEAKER_01
It could be faster and slower.
2
SPEAKER_01
Do you think there could be breakthroughs?
2
SPEAKER_01
And what kind of breakthroughs might there be that completely change that timeline?
2
SPEAKER_01
Again, not only am I asking you to predict the future, I m asking you to predict breakthroughs that haven t happened yet.
2
SPEAKER_00
I think another way to ask that would be, if I could wave a magic wand, what part of the system would I make work today to accelerate it as quickly as possible?
2
SPEAKER_01
Don t say infrastructure, please don t say infrastructure.
2
SPEAKER_00
No, it s definitely not infrastructure.
2
SPEAKER_00
Uh, it s really that, uh, that perception forecasting capability.
2
SPEAKER_00
So if, if, if tomorrow you could give me a perfect model of what s happened, what is happening and what will happen for the next five seconds around a vehicle on the roadway, that would accelerate things pretty dramatically.
2
SPEAKER_01
Are you, in terms of staying up at night, are you mostly bothered by cars, pedestrians, or cyclists?
2
SPEAKER_00
So I, I worry most about the vulnerable road users, about the combination of cyclists and cars, right?
2
SPEAKER_00
Just because cyclists and pedestrians, because, you know, they re not in armor, you know, with the cars, they re bigger, they ve got protection for the people.
2
SPEAKER_00
And so the ultimate risk is, is lower there.
2
SPEAKER_00
Whereas a pedestrian or cyclist, they re out in the road and you know, they, they don t have any protection.
2
SPEAKER_00
And so, you know, we need to pay extra attention to that.
2
SPEAKER_01
Do you think about a very difficult technical challenge of the fact that pedestrians, if you try to protect pedestrians by being careful and slow, they ll take advantage of that.
2
SPEAKER_01
So the game theoretic dance, does that worry you of how, from a technical perspective, how we solve that?
2
SPEAKER_01
because as humans, the way we solve that is kind of nudge our way through the pedestrians, which doesn t feel from a technical perspective as a, uh, appropriate algorithm.
2
SPEAKER_01
Uh, but do you think about how we solve that problem?
2
SPEAKER_00
Yeah, I think, I think there s, there s, I think that was actually, there s two different concepts there.
2
SPEAKER_00
So one is, am I worried that because these vehicles are self-driving people kind of step in the road and take advantage of them?
2
SPEAKER_00
And
2
SPEAKER_00
I ve heard this and I don t really believe it because if I m driving down the road and somebody steps in front of me, I m going to stop.
2
SPEAKER_00
Right, like even if I m annoyed, I m not going to just drive through a person stood in the road.
2
SPEAKER_00
And so I think today people can take advantage of this.
2
SPEAKER_00
And you do see some people do it.
2
SPEAKER_00
I guess there s an incremental risk because maybe they have lower confidence that I m going to see them than they might have for an automated vehicle.
2
SPEAKER_00
And so maybe that shifts it a little bit.
2
SPEAKER_00
But I think people don t want to get hit by cars.
2
SPEAKER_00
And so I think that I m not that worried about people walking out of the 101 and creating chaos more than they would today.
2
SPEAKER_00
regarding kind of the nudging through a big stream of pedestrians leaving a concert or something, I think that is further down the technology pipeline.
2
SPEAKER_00
I think that you re right, that s tricky.
2
SPEAKER_00
I don t think it s necessarily
2
SPEAKER_00
I think the algorithm people use for this is pretty simple.
2
SPEAKER_00
It s kind of just move forward slowly.
2
SPEAKER_00
And if somebody is really close, then stop.
2
SPEAKER_00
And I think that that probably can be replicated pretty easily.
2
SPEAKER_00
And particularly given that you don t do this at 30 miles an hour, you do it at 1, that even in those situations, the risk is relatively minimal.
2
SPEAKER_00
But it s not something we re thinking about in any serious way.
2
SPEAKER_01
And probably that s less an algorithm problem and more creating a human experience.
2
SPEAKER_01
So the HCI people that create a visual display that you re pleasantly as a pedestrian nudged out of the way.
2
SPEAKER_01
That s an experience problem, not an algorithm problem.
2
SPEAKER_01
Who s the main competitor to Aurora today?
2
SPEAKER_01
And how do you out-compete them in the long run?
2
SPEAKER_00
So we really focus a lot of what we re doing here.
2
SPEAKER_00
I think that, you know, I ve said this a few times that this is a huge difficult problem and it s great that a bunch of companies are tackling it because I think it s so important for society that somebody gets there.
2
SPEAKER_00
So we, you know, we re, we don t spend a whole lot of time thinking tactically about who s out there and, and how do we beat that, that, that person individually.
2
SPEAKER_00
Um, what are we trying to do to, to go faster ultimately?
2
SPEAKER_00
Uh, well, part of it is the leisure team we have has got pretty tremendous experience.
2
SPEAKER_00
Um, and so we kind of understand the landscape and understand where the cul-de-sacs are to some degree.
2
SPEAKER_00
And, you know, we try and avoid those.
2
SPEAKER_00
I think there s a part of it, just this great team we ve built.
2
SPEAKER_00
This is a technology and a company that people believe in the mission of, and so it allows us to attract just awesome people to go work.
2
SPEAKER_00
We ve got a culture, I think, that people appreciate that allows them to focus, allows them to really spend time solving problems, and I think that keeps them energized.
2
SPEAKER_00
And then we ve invested hard, invested heavily in the infrastructure and architectures that we think will ultimately accelerate us.
2
SPEAKER_00
So because of the folks we re able to bring in early on, because of the great investors we have, we don t spend all of our time doing demos and kind of leaping from one demo to the next.
2
SPEAKER_00
We ve been given the freedom to invest in
2
SPEAKER_00
infrastructure to do machine learning, infrastructure to pull data from our on-road testing, infrastructure to use that to accelerate engineering.
2
SPEAKER_00
And I think that that early investment and continuing investment in those kind of tools will ultimately allow us to accelerate and do something pretty incredible.
2
SPEAKER_01
Chris, beautifully put.
2
SPEAKER_01
It s a good place to end.
2
SPEAKER_01
Thank you so much for talking today.
2
SPEAKER_00
Thank you very much.
2
SPEAKER_00
Really enjoyed it.