Drago Anguelov is a Principal Scientist at Waymo, developing and applying machine learning methods for autonomous vehicle perception and, more generally, in computer vision and robotics. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo.

INFO:
Website: https://deeplearning.mit.edu
GitHub: https://github.com/lexfridman/mit-deep-learning
Playlist: http://bit.ly/2S1MVdy

OUTLINE:
0:00 - Introduction
0:47 - Background
1:31 - Waymo story (2009 to today)
4:31 - Long tail of events
8:55 - Perception, prediction, and planning
14:54 - Machine learning at scale
26:43 - Addressing the limits of machine learning
29:38 - Large-scale testing
50:51 - Scaling to dozens and hundreds of cities
54:35 - Q&A

CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman

All right, welcome back to 6.094, Deep Learning for Self-Driving Cars.

Today we have Drago Anghielov, Principal Scientist at Waymo.

Aside from having the coolest name in autonomous driving, Drago has done a lot of excellent work in developing and applying machine learning methods to autonomous vehicle perception and more generally in computer vision robotics.

He's now helping Waymo lead the world in autonomous driving.

10 plus million miles achieved autonomously to date, which is an incredible accomplishment.

So it's exciting to have Drago here with us to speak.

I will tell you a bit about our work and the exciting nature of self-driving and the problem and our solutions.

So my talk is called Taming the Long Tail of Autonomous Driving Challenges.

My background is in perception and robotics.

I did a PhD at Stanford with Daphne Koller and worked closely with 1 of the pioneers in the space, Professor Sebastian Thrun.

I spent 8 years at Google doing research on perception, also working on Street View, and developing deep models for detection, neural net architectures.

I was heading the 3D perception team at Zooks.

We built another perception system for autonomous driving and I've been leading the research team at Waymo most recently.

So I want to tell you a little bit about Waymo when we start.

Waymo, actually, this month has its 10-year anniversary.

It started when Sebastian Thrun convinced the Google leadership to try an exciting new moonshot.

And the goal that they set for themselves was to drive 10 different segments that were 100 miles long.

And later that year, they succeeded and drove an order of magnitude more than anyone has ever driven.

In 2015, we brought this car to the road.

It was built ground up as a study in what fully driverless mobility would be like.

In 2015, we put this vehicle in Austin and it completed the world's first fully autonomous ride on public roads.

And the person inside this car is a fan of the project and he is blind.

So we did not want this to be just a demo fully driverless experience.

We worked hard and in 2017 we launched a fleet of fully self-driving vehicles on the streets in Phoenix metro area.

And we have been driverless, fully driverless operations ever since.

So I wanted to give you a feel for what fully driverless experience is like.

Last year we launched our first commercial service in the metro area of Phoenix.

There people can call away on their phone, it can come pick them up and help them with errands or go to school.

We've been already learning a lot from these customers, and we're looking to grow and expand the service and bring it to more people.

So in the process of growing the service, we have driven 10 million miles on public road, as Lex said, and driverlessly, and more also with human drivers to collect data.

And we've driven all kinds of scenarios, cities, capturing a diverse set of conditions and a diverse set of situations from which we develop our systems.

And So in this talk I want to tell you I mean about the long tail of events.

This is all the things we need to handle to enable truly self-driverless future and I guess all the problems that come with this and offer some solutions and show you how Waymo has been thinking about these issues.

So as we drove 10 million miles, of course we still find scenarios, new ones that we have not seen before and we still keep collecting them.

And so when you think about self-driving vehicles, they need to have the following properties.

It needs to be able to handle the entire task of driving.

So it cannot just do a subset and remove the human operator from the vehicle.

And also all of these tasks obviously need to do well and safely.

And that is the requirement to achieving self-driving at scale.

And when you think about this now, the question is, well, how many of these capabilities and how many scenarios do you really need to handle?

Well, it turns out, well, the world is quite diverse and complicated and there is a lot of rare situations and all of them need to be handled well.

It's 1 type of effort to get yourself driving for the common cases, and then it's another effort to tame the rest, and they really, really matter.

For example, this is us driving in the street.

And let's see if you can tell what is unusual in this video.

So there's a bicyclist and he's carrying a stop sign.

And I don't know where he picked it up, but it's certainly not a stop sign we need to stop for, unlike others, right?

This is another case where we are happily staying there and then the vehicle stops and a big pile of poles comes our way, right?

And you need to potentially understand that and learn to avoid it.

Generally, well, different types of objects can fall on the road.

1 of them is someone closed the lane, put a bunch of cones, and we learn.

And this is our vehicle correctly identifying where it's supposed to be driving between all of these cones and successfully executing it.

And this is something that happens fairly often if you drive a lot.

I think you can understand what happened here.

And you can notice actually, so we hear the siren.

So we have the ability to understand sirens of special vehicles and you can see we hear it, it stop and some guys are much later than us braking at the last moment, letting the emergency vehicle pass.

And here's another scenario potentially I want to show you.

Let's see if you can understand what happened.

So we stopped at, there's a green light, we're about to go, and someone goes at high speed running a red light without any remorse, right?

And we successfully stop and prevent issues.

And so sometimes you have the rules of the way and you have your road, and people don't always abide by them.

And that's something that you don't want to just directly go in front of that person, even if they're breaking the law.

So hopefully with this I convinced you that the situations that can occur are diverse and challenging and there's quite a few of them.

And I want to take you a little bit on the tour of what makes this challenging and then tell you some ways in which we think about it and how we're handling it.

And so to do this, we're gonna delve a little bit more into the main tasks for self-driving, which is perception, prediction, and planning.

And so I'll tell you a little bit about those.

Perception, these are the core AI aspects of the car.

We can talk about others as well in a little bit, but let's focus on these first.

So perception is mapping from sensory inputs and potentially prior knowledge of the environment to a scene representation.

And that scene representation can contain objects, it can contain scene semantics, potentially you can reconstruct a map, you can learn about object relationships and so on.

And perception, the space of things you need to handle in perception is fairly hard.

So you have sensors, the pixels come, lighter points come, or radar scans come, and you have multiple axis of variability in the environment.

They have different types, appearance, pose.

I don't know if you see this well, there are a bunch of people dressed as dinosaurs in this case.

People generally are fairly creative in how they dress.

People come in different poses, and we have seen it all.

There's different environments that these objects appear in.

So there are times of day, seasons, day, night, different, for example, highway environment, suburban street and so on.

And then there's a different variability axis and this is a little more, slightly more abstract.

There are different objects can come in this environment in different configurations and can have different relationships.

And so things like occlusion, there's a guy carrying a big board, there is reflections, there is well, people riding on horses and so on.

Because I just want to show you the space.

So in most cases, you care about most objects and most environments and most reasonable configurations.

That's a space that you need to map from the sensor inputs to a representation that makes sense, and you need to learn this mapping function or represent it somehow.

All right, and so let's go to the next step, which is prediction.

So apart from just understanding what's happening in the world, you need to be able to anticipate and predict what some of the actors in the world are going to do, the actors being mostly people, and people is honestly what makes driving quite challenging.

It's, you know, a vehicle needs to be out there and be a full-fledged traffic scene participant.

And this anticipation of agent behavior sometimes needs to be fairly long-term.

So sometimes when you want to make a decision, you want to validate or convince yourself, it does not interfere with what anyone else is going to do, and it can go from 1 second to maybe 10 seconds or more.

So what goes into anticipating the future?

Someone's, I'm going this way, maybe I will continue going there.

Maybe I'm very aggressively walking and maybe I'm more likely to do aggressive motions in the future.

I'll probably stay here and continue, even though stranger things have happened.

And of course there's subtle appearance cues.

So for example, if a person's watching our vehicle and moving towards them, we can be fairly confident they're paying attention and not going to do anything particularly dangerous.

If someone's not paying attention or being distracted or there is a person in the car waving at us, various gestures, cues, the blinkers on the vehicles, these are all signals and subtle signals that we need to understand in order to be able to behave well.

And last but not least, even when you predict how other agents behave, agents also are affected by the other agents in the environment as well.

So I'll give you a short example of this.

I think this is 1 of the issues that really needs to be thought about.

So here's a case, our Waymo vehicle is driving, and there is 2 bicyclists in red going around a parked car, And what happens is we correctly anticipate that as they bike, they will go around the car and we slow down and let them pass, right?

So we're reasoning that they will interact with the parked car.

This is the prediction, our most likely prediction for the rear bicyclists.

We anticipate that they will do this and we correctly handle this case.

Typically ends up in control commands to the vehicle, accelerate, slow down, steer the wheel.

And you need to generate behavior that ultimately has several properties to it, and it's important to think of them, which is safe, safety comes first, comfortable for the passengers, and also sends the right signals to the other traffic participants, because they can interact with you and they will react to your actions, so you need to be mindful.

So you need to trade all of these in a reasonable way.

And it, and it can be fairly sophisticated reasoning in complex environments.

This is, this is a complex I think school gathering, there's bicyclists trailing us, vehicles really closely hemmed within us, a bunch of pedestrians, and we need to make progress, and here is us, we're driving, and reasonably well, in crowded scenes.

And that is part of the prerequisite of bringing this technology to, you know, the dense urban environments.

I think when you have these complicated models and systems, Machine learning is a really great tool to model complex actions, complex mapping functions, features, right?

And we've been doing this, I mean, we're not the only ones, so obviously this, this is now a machine learning revolution.

And machine learning is permeating all parts of the way in Mostak.

All of these systems that I'm talking about, it helps us perceive the world, it helps us make decisions about what others are going to do, it helps us make our own decisions.

And machine learning is a tool to handle the long tail.

Right, and I'll tell you a little more on this how.

So I have this allegory about machine learning that I like to think about.

So there is a classical system and there is a machine learn system.

And to me a classical system, and I've been there, I've done well, early machine learning systems also can be a bit classical.

You're the artisan, you're the expert, you have your tools, and you need to build this product, and you have your craft, and you go and take your tools and build it, right?

And it can fairly quickly get something reasonable, but then it's harder to change, it's harder to evolve.

If you learn new things, now you need to go back and maybe the tools don't quite fit, and you need to essentially keep tweaking it, and it starts becoming, the more complicated the product becomes, the harder it is to do.

And machine learning, modern machine learning, is like a factory.

So machine learning, you build the factory, which is the machine learning infrastructure, and then you feed data in this factory and get nice models that solve your problems, right?

And so, kind of infrastructure is at the heart of this new paradigm.

All right, Once you do it, now you can iterate, it's scalable, just keep the right data, keep feeding the machine, keeps giving you good models.

So what is that ML factory for self-driving models?

We have a software release, we put it on the vehicle, we're able to drive.

We drive, we collect data, we collect it, and we store it.

Then we select some parts of this data, and we send it to labelers.

The labelers label parts of the data that we find interesting, and that's a knowledge that we want to extract from the data.

These are the labels, the annotations, the results we want for our models.

Then what we're going to do is we're gonna train machine learning models on this data.

After we have the models, we will do testing and validation, validate that they're good to put on our vehicles.

And once they're good to put on our vehicles, we go and collect more data.

And then the process starts going again and again.

So you collect more data, now you select new data that you have not selected before.

You add it to your data set, you keep training the model.

And you iterate, iterate, iterate, it's a nice scalable setup.

And at Waymo, we have the beautiful advantage to be really well set up with regards to the machine learning infrastructure.

And I'll tell you a bit about its ingredients and how we go about it.

So ingredient 1 is computing software infrastructure, and we're part of Alphabet, Google, and we are able to, first of all, leverage TensorFlow, the deep learning framework.

We have access to the experts that wrote TensorFlow and know it in depth.

We have data centers to run large-scale parallel compute and also train models.

We have specialized hardware for training models, which make it cheaper and more affordable and faster so you can iterate better.

Ingredient 2 is high-quality labeled data.

We have the scale to collect and store hundreds and thousands and more miles to millions of miles.

And just collecting and storing 10 millions miles is not necessarily the best thing you can do, because there is a decreasing utility to the data.

So most of the data comes from common scenarios and maybe you're already good at them, and that's where the long tail comes.

So It's really important how you select the data.

So this is important part of this pipeline.

So while you're running release on the vehicle, we have a bunch of models, we have a bunch of understanding about the world, and we annotate the data as we go, and we can use this knowledge to decide what data is interesting, how to store it, which data we can potentially even ignore.

So then once we do that, again, we need to be very careful how to select data.

We want to select data, for example, that are interesting in some way and complement, capture these long tail cases that we potentially may not be doing so well on.

So for this, we have active learning and data mining pipelines.

Given exemplars, find the rare examples, look for parts of your system which are uncertain or, you know, inconsistent over time and go and label those cases.

Last but not least, we also produce auto-labels.

Well, when you collect data, you also see the future for many of the objects, what they did.

And so because of that, now knowing the past and the future, you can annotate your data better and then go back to your model that does not know the future and try to replicate that with that model.

And so you need to do all of this as part of the system.

Ingredient number 3, high quality models.

We're part of larger Alphabet and Google and DeepMind and generally Alphabet is the leader in AI.

When I was at Google, we were very early on the deep learning revolution.

I happened to have the chance to be there at the time.

It was 2013 when I got on to do deep learning and a lot of things were not understood and we were there working on it earlier than most people.

And so through that we had the opportunity and the chance to develop some of the, in my time, the team I managed, we invented neural net architecture like Inception, which became popular later.

We invented at the time the state-of-the-art object detection, fast object detector called SSD.

And now if you go to the conferences, Google and DeepMind, the leaders in perception and reinforcement learning and smart agents.

And there is like state of the art, say semantic segmentation networks, pose estimation and so on.

The object detection of course goes without saying.

And so we collaborate with Google and DeepMind on projects improving our models.

And so this is my factory for self-driving models, and I want to tell you something that kind of captures all of these ideas, infrastructure, data, and models in 1.

This is a project we did recently, and today we put online in our blog about automatic machine learning for tuning and adjusting architectures of neural networks.

So there is a team at Google working on AutoML, automatic machine learning.

And usually networks themselves have complex architecture.

They're crafted by practitioners to artisans of networks in some way.

And sometimes, you know, we have very high latency constraints in the models.

It takes often people months to find the right architecture that's most performant, low latency, and so on.

And so there's a way to offload this work to the machines.

You can have machines themselves, once you've exposed the problem, go and find you a good network architecture that's both low latency and high performance.

And we drive in a lot of scenarios and we, as we keep collecting data and finding new cities or new examples, the architectures may change and we want to easily find that and keep evolving that without too much effort, right?

So we worked with the Google researchers and they had a strong work where they invented, well, they developed a system that searched the space of architectures and found a set of components of neural networks.

It's a small sub network called NASCEL and this is a diagram of a NASCEL.

It's a set of layers put together that you can then replicate in the network to build a larger network.

And they discovered in a small vision data set, it was called CIFAR-10.

It's from the early days of deep learning.

And you can quickly train models and, and explore the large search space.

So the first thing we did is we took some problems in, that we have for our stack, 1 of them being lighter segmentation.

So you have a map representation and some lighter points.

And, you essentially segment the lighter points.

You say this point is part of a vehicle, that point is part of vegetation and so on.

So what we first did at Waymo is we explored several hundred NAS cell combinations to see what performs better on this task.

And we saw that 1 of 2 things happened for the various versions that we found.

1 of them is, we can find models with similar quality but much lower latency, and less compute.

And then there is, models of a bit higher quality at the same latency.

So essentially, we found better models than the human engineers did.

And similar results were obtained for other problems, lane detection as well, with this transfer learning approach.

Of course, you can also do end-to-end architecture search.

So there's no reason why what was found on CIFAR-10 is best suited for our more specialized problems.

And so we went about this more from the ground up.

So let's find exactly deeper search, much, much larger space, not limited to the NAS cells themselves.

And so the way to do this is because our networks are trained on quite a lot of data and take quite a while to converge and it takes some compute, we went and defined the proxy task.

See all Lex Fridman transcripts on Youtube

Drago Anguelov (Waymo) - MIT Self-Driving Cars