This is a clip from a conversation with George Hotz on the Artificial Intelligence podcast. You can watch the full conversation here: http://bit.ly/2YLIPom If you enjoy these, consider subscribing, sharing, and commenting below.

Full episode: http://bit.ly/2YLIPom
Full episodes playlist: http://bit.ly/2EcbaKf
Clips playlist: http://bit.ly/2JYkbfZ
Podcast website: https://lexfridman.com/ai

George Hotz is the founder of Comma.ai, a machine learning based vehicle automation company. He is an outspoken personality in the field of AI and technology in general. He first gained recognition for being the first person to carrier-unlock an iPhone, and since then has done quite a few interesting things at the intersection of hardware and software.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

So the way you leapfrog, right, is you come up with an idea or you take a direction, perhaps secretly, that the other people aren't taking.

I don't know Aurora, Zoox is the same stack as well.

the same DARPA Urban Challenge code base.

So the question is, do you think there's a room for brilliance and innovation there that will change everything?

Like say, okay, so I'll give you examples.

It could be if revolution and mapping, for example, that allow you to map things, do HD maps of the whole world, all weather conditions somehow really well, or revolution in simulation, to where all the, what you said before becomes incorrect.

What I said before about, oh, they actually get the whole thing.

I'll say this about, we divide driving into 3 problems.

And I actually haven't solved the third yet, but I have an idea how to do it.

The static driving problem is assuming you are the only car on the road.

And this problem can be solved 100% with mapping and localization.

If all you have to deal with is the static problem, and you can statically schedule your machines, right?

It's the same as statically scheduling processes.

You can statically schedule your tractors to never hit each other on their paths, right?

Maps only helps you with the static driving problem.

The question about static driving, you just made it sound like it's really easy.

Well, because the whole drifting out of lane.

When Tesla drifts out of lane, it's failing on the fundamental static driving problem.

The static driving problem is not easy for the world.

The static driving problem is easy for 1 route.

1 route and 1 weather condition with 1 state of lane markings and like no deterioration, no cracks in the road.

I'm assuming you have a perfect localizer.

So that's solved for the weather condition and the lane marking condition.

Perfect localizers are not that hard to build.

Like, use LIDAR to build a perfect localizer.

Building a perfect localizer without LIDAR...

You can get 10 centimeters without LiDAR, you can get 1 centimeter with LiDAR.

I'm not even concerned about the 1 or 10 centimeters, I'm concerned if every once in a while you're just way off.

Yeah, so this is why you have to carefully make sure you're always tracking your position.

You wanna use LiDAR camera fusion, but you can get the reliability of that system up to 100,000 miles, and then you write some fallback condition where it's not that bad if you're way off, right?

I think that you can get it to the point, it's like as will D that you're never in a case where you're way off and you don't know it.

We can, Especially with LIDAR and good HD maps, you can solve that problem.

No, you just disagree with the word easy.

It's not as challenging as the other ones.

Well, it's, okay, Maybe it's obvious how to solve it.

And a lot of people don't even think about the third 1 and even see it as different from the second 1.

The second 1 is like, say there's an obvious example, it's like a car stopped at a red light.

You can't have that car in your map because You don't know whether that car is going to be there or not.

So you have to detect that car in real time.

And then you have to do the appropriate action.

And you have to predict what that car will do.

This involves, again, you're gonna need models of other people's behavior.

Are you including in that, I don't wanna step on the third 1, but Are you including in that your influence and people?

I just talked to Judea Pearl who's obsessed with counterfactuals.

The counterfactual, oh yeah, yeah, I read his books.

So the static and the dynamic, our approach right now for lateral will scale completely to the static and dynamic.

The counterfactual, the only way I have to do it yet, the thing that I want to do once we have all of these cars is I want to do reinforcement learning on the world.

I'm always going to turn the exploiter up to max, I'm not going to have them explore, but the only real way to get at the counterfactual is to do reinforcement learning because the other agents are humans.


Speaker 0: So the way you leapfrog, right, is you come up with an idea or you take a direction, perhaps secretly, that the other people aren't taking. And so, the Cruz, Waymo, even Aurora.

Speaker 1: I don't know Aurora, Zoox is the same stack as well. They're all the same code base even.

Speaker 0: They're all

Speaker 1: the same DARPA Urban Challenge code base.

Speaker 0: So the question is, do you think there's a room for brilliance and innovation there that will change everything? Like say, okay, so I'll give you examples. It could be if revolution and mapping, for example, that allow you to map things, do HD maps of the whole world, all weather conditions somehow really well, or revolution in simulation, to where all the, what you said before becomes incorrect. That kind of thing. Any room for breakthrough innovation?

Speaker 1: What I said before about, oh, they actually get the whole thing. I'll say this about, we divide driving into 3 problems. And I actually haven't solved the third yet, but I have an idea how to do it. So there's the static. The static driving problem is assuming you are the only car on the road.

And this problem can be solved 100% with mapping and localization. This is why farms work the way they do. If all you have to deal with is the static problem, and you can statically schedule your machines, right? It's the same as statically scheduling processes. You can statically schedule your tractors to never hit each other on their paths, right?

Because you know the speed they go at. So that's the static driving problem. Maps only helps you with the static driving problem.

Speaker 0: Yeah. The question about static driving, you just made it sound like it's really easy.

Speaker 1: Static driving is really easy.

Speaker 0: How easy? How hard? Well, because the whole drifting out of lane. When Tesla drifts out of lane, it's failing on the fundamental static driving problem.

Speaker 1: Tesla is drifting out of lane? The static driving problem is not easy for the world. The static driving problem is easy for 1 route.

Speaker 0: 1 route and 1 weather condition with 1 state of lane markings and like no deterioration, no cracks in the road.

Speaker 1: I'm assuming you have a perfect localizer. So that's solved for the weather condition and the lane marking condition.

Speaker 0: But that's the problem. How do you have a perfect localizer?

Speaker 1: You can build... Perfect localizers are not that hard to build.

Speaker 0: Okay, come on now. With LIDAR? With LIDAR, yeah. Oh, with LIDAR, okay.

Speaker 1: With LIDAR, yeah. But you use LIDAR, right? Like, use LIDAR to build a perfect localizer. Building a perfect localizer without LIDAR... It's going to be hard.

You can get 10 centimeters without LiDAR, you can get 1 centimeter with LiDAR.

Speaker 0: I'm not even concerned about the 1 or 10 centimeters, I'm concerned if every once in a while you're just way off.

Speaker 1: Yeah, so this is why you have to carefully make sure you're always tracking your position. You wanna use LiDAR camera fusion, but you can get the reliability of that system up to 100,000 miles, and then you write some fallback condition where it's not that bad if you're way off, right? I think that you can get it to the point, it's like as will D that you're never in a case where you're way off and you don't know it.

Speaker 0: Yeah, okay, so this is brilliant. So that's the static. Static. We can, Especially with LIDAR and good HD maps, you can solve that problem. Easy.

No, you just disagree with the word easy.

Speaker 1: The static problem's so easy. It's very

Speaker 0: typical for you to say something's easy. I got it. It's not as challenging as the other ones. Okay.

Speaker 1: Well, it's, okay, Maybe it's obvious how to solve it. The third one's the hardest. Where do I get? And a lot of people don't even think about the third 1 and even see it as different from the second 1. So the second 1 is dynamic.

The second 1 is like, say there's an obvious example, it's like a car stopped at a red light. You can't have that car in your map because You don't know whether that car is going to be there or not. So you have to detect that car in real time. And then you have to do the appropriate action. Also, that car is not a fixed object.

That car may move. And you have to predict what that car will do. So this is the dynamic problem. So you have to deal with this. This involves, again, you're gonna need models of other people's behavior.

Speaker 0: Are you including in that, I don't wanna step on the third 1, but Are you including in that your influence and people?

Speaker 1: Ah, that's the third 1. Okay. That's the third 1. We call it the counterfactual. Yeah, brilliant.

And that.

Speaker 0: I just talked to Judea Pearl who's obsessed with counterfactuals.

Speaker 1: The counterfactual, oh yeah, yeah, I read his books. So the static and the dynamic, our approach right now for lateral will scale completely to the static and dynamic. The counterfactual, the only way I have to do it yet, the thing that I want to do once we have all of these cars is I want to do reinforcement learning on the world. I'm always going to turn the exploiter up to max, I'm not going to have them explore, but the only real way to get at the counterfactual is to do reinforcement learning because the other agents are humans.

See all Lex Fridman transcripts on Youtube

George Hotz: 3 Problems of Autonomous Driving: Static, Dynamic, Counterfactual | AI Podcast Clips