He was the CTO of the Google self-driving car team, a key engineer and leader behind the Carnegie Mellon University autonomous vehicle entries in the DARPA Grand Challenges, and the winner of the DARPA Urban Challenge.

Today, he's the CEO of Aurora Innovation, an autonomous vehicle software company he started with Sterling Anderson, who was the former director of Tesla Autopilot, and Drew Bagnell, Uber's former autonomy and perception lead.

Chris is 1 of the top roboticists and autonomous vehicle experts in the world and a long time voice of reason in a space that is shrouded in both mystery and hype.

He both acknowledges the incredible challenges involved in solving the problem of autonomous driving and is working hard to solve it.

This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube, give it 5 stars on iTunes, support it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D-M-A-N.

And now, here's my conversation with Chris Armstron.

You were part of both the DARPA Grand Challenge and the DARPA Urban Challenge teams at CMU with Red Whitaker.

What technical or philosophical things have you learned from these races?

I think the high order bit was that it could be done.

I think that was the thing that was incredible about the first, the grand challenges.

That I remember, I was a grad student at Carnegie Mellon.

It was this dichotomy of, It seemed really hard, so that would be cool and interesting.

But at the time, we were the only robotics institute around.

And so if we went into it and fell on our faces, that would be embarrassing.

So I think just having the will to go do it, to try to do this thing that at the time was marked as, you know, darn near impossible, and then after a couple of tries be able to actually make it happen.

I think that was, you know, that was really exciting.

But at which point did you believe it was possible?

Did you, from the very beginning, did you personally, because you're 1 of the lead engineers, you actually had to do a lot of the work.

Yeah, I was the technical director there and did a lot of the work along with a bunch of other really good people.

Right, like why would you go do something you thought was completely impossible?

We didn't know how we're going to be able to do it.

We didn't know if we'd be able to do it the first time.

I think there's a certain benefit to naivete, right?

That if you don't know how hard something really is, you try different things and it gives you an opportunity that others who are wiser maybe don't have.

Mechanical, sensors, hardware, software, algorithms for mapping, localization, just general perception and control, but the hardware, software, first of all.

I think that's the joy of this field is that it's all hard and that you have to be good at each part of it.

So for the first for the urban challenges, if I look back at it from today, it should be easy today.

That it was a static world, there weren't other actors moving through it, that is what that means.

It was out in the desert, so you get really good GPS.

So that went, and we could map it roughly.

And so in retrospect, now it's within the realm of things we could do back then.

Just actually getting the vehicle, and there's a bunch of engineering work to get the vehicle so that we could control it and drive it.

That's, you know, that's still a pain today, but it was even more so back then.

And then the uncertainty of exactly what they wanted us to do was part of the challenge as well.

Right, you didn't actually know the track heading in here.

You knew approximately, but you didn't actually know the route, the route that's gonna be taken.

We didn't even really, the way the rules had been described, you had to kind of guess.

So if you think back to that challenge, the idea was that the government would give us, that DARPA would give us a set of waypoints and kind of the width that you had to stay within between the line that went between each of those waypoints.

And so the most devious thing they could have done is set a kilometer-wide corridor across a field of scrub brush and rocks and said, you know, go figure it out.

Fortunately, it really, it turned into basically driving along a set of trails, which is much more relevant to the application they were looking for.

But no, it was a hell of a thing back in the day.

So the legend, Red, was kind of leading that effort in terms of just broadly speaking.

What have you learned from Red about leadership?

1 is go and try those really hard things.

That's where there is an incredible opportunity.

I think the other big 1, though, is to see people for who they can be, not who they are.

It's 1 of the things that I actually, 1 of the deepest lessons I learned from Red was that he would look at undergraduates or graduate students and empower them to be leaders, to have responsibility, to do great things, that I think another person might look at them and think, oh, well, that's just an undergraduate student.

And so I think that kind of trust, but verify, have confidence in what people can become, I think is a really powerful thing.

So through that, let's just like fast forward through the history.

Can you maybe talk through the technical evolution of autonomous vehicle systems from the first 2 grand challenges to the urban challenge to today?

Are there major shifts in your mind or is it the same kind of technology just made more robust?

I think there's been some big, big steps.

So for the grand challenge, the real technology that unlocked that was HD mapping.

Prior to that, a lot of the off-road robotics work had been done without any real prior model of what the vehicle was going to encounter.

And so that innovation, that the fact that we could get, you know, decimeter resolution models was really a big deal.

And that allowed us to kind of bound the complexity of the driving problem the vehicle had and allowed it to operate at speed because we could assume things about the environment that it was going to encounter.

For the urban challenge, 1 of the big technological innovations there was the multi-beam LIDAR and being able to generate high resolution, mid to long range 3D models the world and use that for understanding the world around the vehicle.

And that was really a game-changing technology.

In parallel with that, we saw a bunch of other technologies that had been kind of converging half their day in the sun.

So Bayesian estimation had been, SLAM had been a big field in robotics.

You would go to a conference a couple of years before that and every paper would effectively have SLAM somewhere in it.

And so seeing that, those Bayesian estimation techniques play out on a very visible stage, I thought that was pretty exciting to see.

And mostly SLAM was done based on LIDAR at that time?

Well, yeah, and in fact, we weren't really doing SLAM per se in real time because we had a model ahead of time, we had a roadmap, but we were doing localization.

And we're using the LIDAR or the cameras, depending on who exactly was doing it, to localize to a model of the world.

And I thought that was a big step from kind of naively trusting GPS, INS before that.

And again, like lots of work had been going on in this field.

Certainly this was not doing anything particularly innovative in Slam or in localization, but it was seeing that technology necessary in a real application on a big stage, I thought was very cool.

So for the urban challenge, those are already maps constructed offline?

Did the individual teams do it individually?

So they had their own different approaches there or did everybody kind of share that information at least intuitively?

So DARPA gave all the teams a model of the world, a map.

And then 1 of the things that we had to figure out back then was, and it's still 1 of these things that trips people up today, is actually the coordinate system.

And to so many decimal places, you don't really care about the ellipsoid of the Earth that's being used.

But when you want to get to 10 centimeter or centimeter resolution, you care whether the coordinate system is NADS 83 or WGS 84, Or these are different ways to describe both the kind of non-sphericalness of the Earth, but also kind of the, I think, I can't remember which 1, the tectonic shifts that are happening and how to transform the global data as a function of that.

So, getting a map and then actually matching it to reality to centimeter resolution, that was kind of interesting and fun back then.

So how much work was the perception doing there?

So how much were you relying on localization based on maps without using perception to register to the maps?

And I guess the question is, how advanced was perception at that point?

It's certainly behind where we are today.

We're more than a decade since the urban challenge.

But the core of it was there, that we were tracking vehicles.

We had to do that at 100 plus meter range because we had to merge with other traffic.

We were using Bayesian estimates for state of these vehicles.

We had to deal with a bunch of the problems that you think of today of predicting where that vehicle is going to be a few seconds into the future.

We had to deal with the fact that there were multiple hypotheses for that because a vehicle at an intersection might be going right or it might be going straight or it might be making a left turn.

And we had to deal with the challenge of the fact that our behavior was going to impact the behavior of that other operator.

And, you know, we did a lot of that in relatively naive ways, but it

And so where does that 10 years later, Where does that take us today from that artificial city construction to real cities to the urban environment?

Yeah, I think the biggest thing is that the actors are truly unpredictable.

That most of the time, the drivers on the road, the other road users are out there behaving well, but every once while they're not.

The variety of other vehicles is, you have all of them.

In terms of behavior, in terms of perception, or both?

That we have, back then we didn't have to deal with cyclists, we didn't have to deal with pedestrians, didn't have to deal with traffic lights.

The scale over which that you have to operate is now, is much larger than the airbase that we were thinking about back then.

What do you think is the hardest part about driving?

I'm sure nothing really jumps out at you as 1 thing, but, in, in the jump from the urban challenge to the real world, is there something that's a particular, you foresee as very serious, difficult challenge?

I think the most fundamental difference is that we're doing it for real.

That in that environment, it was both a limited complexity environment because certain actors weren't there, because the roads were maintained, there were barriers keeping people separate from robots at the time.

And it only had to work for 60 miles, which looking at it from 2006, it had to work for 60 miles.

Looking at it from now, we want things that will go and drive for half a million miles.

So how important, you said LiDAR came into the game early on and it's really the primary driver of autonomous vehicles today as a sensor.

So how important is the role of LiDAR in the sensor suite in the near term?

I believe, But I also believe the cameras are essential and I believe the radar is essential.

I think that you really need to use the composition of data from these different sensors if you want the thing to really be robust.

The question I want to ask, let's see if we can untangle it, is what are your thoughts on the Elon Musk provocative statement that LIDAR is a crutch?

That is a kind of, I guess, growing pains, and that much of the perception task can be done with cameras.

So I think it is undeniable that people walk around without lasers in their foreheads, and they can get into vehicles and drive them.

And so there's an existence proof that you can drive using you know passive vision no doubt can't argue with that

So like there's an example that we all go do it, many of us, every day.

But in the same way that the combustion engine was a crutch on the path to an electric vehicle, the same way that any technology ultimately gets replaced by some superior technology in the future.

And Really, the way that I look at this is that the way we get around on the ground, the way that we use transportation is broken.

And that we have this, what was I think the number I saw this morning, 37,000 Americans killed last year on our roads.

And so any technology that we can bring to bear that accelerates this self-driving technology, coming to market and saving lives, is technology we should be using.

And it feels just arbitrary to say, well, I'm not okay with using lasers, because that's whatever, but I am okay with using an 8 megapixel camera, or a 16 megapixel camera.

These are just bits of technology, and we should be taking the best technology from the tool bin that allows us to go and, you know, and solve a problem.

The question I often talk to, well, obviously you do as well, to the sort of automotive companies.

And, you know, if there's 1 word that comes up more often than anything, it's cost and trying to drive cost down.

So while it's true that it's a tragic number, the 37,000, the question is what...

And I'm not the 1 asking this question because I hate this question, but we want to find the cheapest sensor suite that creates a safe vehicle.

In that uncomfortable trade-off, Do you foresee LIDAR coming down in cost in the future, or do you see a day where level 4 autonomy is possible without LIDAR?

I see both of those, but it's really a matter of time.

And I think really, maybe I would talk to the question you asked about the cheapest sensor.

I don't think that's actually what you want.

What you want is a sensor suite that is economically viable.

And then after that, everything is about margin and driving cost out of the system.

What you also want is a sense suite that works.

And so it's great to tell a story about how it would be better to have a self-driving system with a $50 sensor instead of a, you know, a $500 sensor.

But if the $500 sensor makes it work and the $50 sensor doesn't work, you know, who cares?

So long as you can actually, you know, have an economic offer, you know, there's an economic opportunity there.

And the economic opportunity is important because that's how you actually have a sustainable business and that's how you can actually see this come to scale and be out in the world.

And so when I look at LIDAR, I see a technology that has no underlying fundamental expense to it.

It's going to be more expensive than an imager because CMOS processes or FAB processes are dramatically more scalable than mechanical processes, but we still should be able to drive cost down substantially on that side.

And then I also do think that with the right business model, you can absorb more, you know, certainly more cost on the bill of materials.

If the sensor suite works, extra value is provided, thereby you don't need to drive costs down to 0.

You've talked about your intuition that level 2 autonomy is problematic because of the human factor of vigilance, decrement, complacency, overtrust, and so on, just us being human.

We overtrust the system, we start doing even more so partaking in the secondary activities like smartphones and so on.

Have your views evolved on this point in either direction?

So I want to be really careful because sometimes this gets twisted in a way that I certainly didn't intend.

So active safety systems are a really important technology that we should be pursuing and integrating into vehicles.

And there's an opportunity in the near term to reduce accidents, reduce fatalities, and we should be pushing on that.

Level 2 systems are systems where the vehicle is controlling 2 axes.

And I think there are variants of level 2 systems that are supporting the driver that absolutely we should encourage to be out there.

Where I think there's a real challenge is in the the human factors part around this and the misconception from the public around the capability set that that enables and the trust they should have in it.

And that is where I kind of, I am actually incrementally more concerned around level 3 systems and how exactly a level 2 system is marketed and delivered.

And how much effort people have put into those human factors.

So I still believe several things around this.

1 is people will overtrust the technology.

We've seen over the last few weeks a spate of people sleeping in their Tesla.

I watched an episode last night of Trevor Noah talking about this.

And him, this is a smart guy who has a lot of resources at his disposal describing a Tesla as a self-driving car, and that why shouldn't people be sleeping in their Tesla?

It's like, well, because it's not a self-driving car, and it is not intended to be, and these people will almost certainly die at some point or hurt other people.

And so we need to really be thoughtful about how that technology is described and brought to market.

I also think that because of the economic issue, economic challenges we were just talking about, that that technology path will all these level 2 driver assistance systems, that technology path will diverge from the technology path that we need to be on to actually deliver truly self-driving vehicles, ones where you can get in it and sleep and have the equivalent or better safety than a human driver behind the wheel.

Because again, the economics are very different in those 2 worlds.

And so that leads to divergent technology.

So you just don't see the economics of gradually increasing from level 2 and doing so quickly enough to where it doesn't cause safety, critical safety concerns.

You believe that that it needs to diverge at this point into different, basically different routes.

And really that comes back to what are those L2 and L1 systems doing?

And they are driver assistance functions where the people that are marketing that responsibly are being very clear and putting human factors in place such that the driver is actually responsible for the vehicle and that the technology is there to support the driver.

And the safety cases that are built around those are dependent on that driver attention and attentiveness.

And at that point, you can kind of give up to some degree for economic reasons, you can give up on, say, false negatives.

And the way to think about this is for a forklift mitigation braking system, if half the times the driver missed a vehicle in front of it, It hit the brakes and brought the vehicle to a stop.

That would be an incredible, incredible advance in safety on our roads.

But it would mean that if that vehicle wasn't being monitored, it would hit 1 out of 2 cars.

And so economically, that's a perfectly good solution for a driver assistance system.

What you should do at that point, if you can get it to work 50% of the time, is drive the cost out of that so you can get it on as many vehicles as possible.

But driving the cost out of it doesn't drive up performance on the false negative case.

And so you'll continue to not have a technology that could really be available for a self-driven vehicle.

So clearly the communication, and this probably applies to all 4 vehicles as well, the marketing and the communication of what the technology is actually capable of, how hard it is, how easy it is, all that kind of stuff is highly problematic.

So say everybody in the world was perfectly communicated and were made to be completely aware of every single technology out there, what they, what it's able to do.

And now we're maybe getting into philosophical ground.

Is it possible to have a level 2 vehicle where we don't overtrust it?

If people truly understood the risks and internalized it, then sure, you could do that safely.

The people are going to, if the facts are put in front of them, they're going to then combine that with their experience.

And you know, let's say they're, they're using an L2 system and they go up and down the 101 every day and they do that for a month.

And it just worked every day for a month.

At that point, even if you know the statistics, you're like, well, I don't know.

Maybe there's something funny about those.

Maybe they're driving in difficult places.

And the problem is that that sample size that they have, so it's 30 miles up and down, so 60 miles times 30 days, so 60, 180, 1,800 miles.

That's a drop in the bucket compared to the 85 million miles between fatalities.

And so they don't really have a true estimate based on their personal experience of the real risks, but they're going to trust it anyway because it's hard not to.

So even if you start a perfect understanding of the system, your own experience will make it drift.

Over a year, over 2 years, even it doesn't have to be months.

And I think that as this technology moves from what I would say is kind of the more technology savvy ownership group to the mass market, you may be able to have some of those folks who are really familiar with technology, they may be able to internalize it better.

And your immunization against this false risk assessment might last longer.

But as folks who aren't as savvy about that read the material and they compare that to their personal experience, I think there it's going to move more quickly.

So your work, the program that you've created at Google and now at Aurora, is focused more on the second path of creating full autonomy.

So it's such a fascinating, I think it's 1 of the most interesting AI problems of the century, right?

I just talked to a lot of people, just regular people, I don't know, my mom, about autonomous vehicles.

And you begin to grapple with ideas of giving your life control over to a machine is philosophically interesting.

How do you think we demonstrate, you've spoken about metrics in the past.

How do you think we demonstrate to the world that an autonomous vehicle, an Aurora system is safe?

This is 1 where it's difficult because there isn't a soundbite answer.

That we have to show a combination of work that was done diligently and thoughtfully.

And this is where something like a functional safety process is part of that.

It's like, here's the way we did the work.

So if you believe what we said about this is the way we did it, then you can have some confidence that we were thorough in the engineering work we put into the system.

And then on top of that, to kind of demonstrate that we weren't just thorough, we were actually good at what we did.

See all Lex Fridman transcripts on Youtube

Chris Urmson: Self-Driving Cars at Aurora, Google, CMU, and DARPA | Lex Fridman Podcast #28