Karl Iagnemma is the President of Aptiv Autonomous Mobility. Oscar Beijbom is their Machine Learning Lead. In 2013, Karl co-founded nuTonomy, a Boston-based autonomous vehicle company that was sold to Delphi (Aptiv) in 2017 for $400 million. Now as part of Aptiv, Karl and team are one of the leaders in autonomous vehicle development and deployment, with cars on the roads in several sites in the US. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo.

INFO:
Website: https://deeplearning.mit.edu
GitHub: https://github.com/lexfridman/mit-deep-learning
Playlist: http://bit.ly/2S1MVdy

OUTLINE:
0:00 - Introduction to Karl Iagnemma and Oscar Beijbom
1:00 - Karl - Aptiv Background
10:18 - Dimensions of Safety for AVs
12:47 - Trusting neural networks behind the wheel
15:07 - Validation of black-box systems
17:50 - Trusting the data
19:27 - Trusting the algorithms
22:27 - Safety architecture for neural networks
25:20 - Engineering is inching closer to the natural sciences
25:57 - Oscar - DL for 3D Detection
30:06 - PointPillars
39:51 - nuScenes - a dataset for multimodal 3d object detection
43:17 - Q&A

CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman

right, welcome back to 6S094, Deep Learning for Self-Driving Cars.

Today we have Carl and Yema, and Oscar Bayboom from Aptiv.

Carl is the president of Aptiv Autonomous Mobility, where Oscar is the machine learning lead.

Carl founded Newtonomy, as many of you know, in 2013.

It's a Boston-based autonomous vehicle company.

and now as part of Aptiv, Carl and team are 1 of the leaders in autonomous vehicle development and deployment with cars on roads all over the United States, several sites.

But most importantly, Carl is MIT through and through, as also some of you may know, getting his PhD here.

He led a robotics group here as a research scientist for many years.

So it's really a pleasure to have both Carl and Oscar with us today.

Very impressed that you guys are here during IAP.

My course load during IAP was usually ice skating and sometimes like there was a wine tasting course.

This was now almost 20 years ago and that was pretty much it.

So you guys are here to learn something so I'm gonna do my best and try something radical actually.

Since I'm president now of Aptiv Autonomous Driving I'm not allowed to talk about anything technical or interesting.

I'm gonna flout that a little bit and raise some topics that we think about that I think are interesting questions to keep in the back of your mind as you're thinking about deep learning and autonomous driving.

And then Oscar will actually present some real life technology and some of the work that he has been doing.

Some of the work that he and his outstanding team have been doing around machine learning based detectors for the perception problem.

So let me first introduce Aptiv a little bit because people usually ask me, like what's an Aptiv when I say I work for Aptiv?

Aptiv's actually been around for a long time, but in a different form.

Aptiv was previously Delphi Technologies, which was previously part of General Motors.

Some of you may have heard of Delphi, Aptiv spun from Delphi about 14 months ago.

They're an automotive company that industrializes technology.

Essentially, they take software and hardware, they industrialize it and put it on cars so it can run for many, many hundreds of thousands of miles without failing.

Which is a useful thing when we think about autonomous driving.

So the themes for Aptiv, they develop what they say is safer, greener, and more connected solutions.

Safer means safety systems, active safety, autonomous driving systems of the type that we're building.

Greener, systems to enable electrification and kind of green vehicles, and then more connected connectivity solutions, both within the vehicle, transmitting data around the vehicle, and then externally, wireless communication.

All of these things, as you can imagine, feed very, very nicely into the future transportation systems that the software will actually only be a part of.

So Aptiv is in a really interesting spot when you think about the future of autonomous driving.

And to give you a sense of scale, still kind of amazes me.

The biggest my research group ever was at MIT was like 18, 18 people.

employees, so significant sized organization, about a $13 billion company by revenue in about 50 countries around the world.

My group's about 700 people, so of which Oscar is 1 very important person.

We're about 700 working on autonomous driving.

We've got about 120 cars on the road in different countries, and I'll show you some examples of that.

But first, let me take a trip down memory lane and show you a couple of snapshots about where we were not too long ago, kind of as a community, but also me personally.

And this will either inspire or horrify you, I'm not sure which.

there were groups driving around with cars, like running blade servers in the trunk that were generating so much heat, you had to install another air conditioner, which then was drawing so much power, you had to add another alternator, and then kind of rinse and repeat.

So it wasn't a great situation, but people did enough algorithmically, computationally to enable these cars, and this is the DARPA Urban Challenge for those of you that may be familiar, to enable these cars to do something useful and interesting on a closed course.

And it kind of convinced enough people that given enough devotion of thought and resources that this might actually become a real thing someday.

So I was 1 of those people that got convinced.

this is now, I'm going to crib from my co-founder Emilio, who was a former MIT faculty member in AeroAstro.

Emilio started up an operation in Singapore through SMART, who some of you have probably worked with.

That's James, who looks really young in that picture.

He was 1 of Emilio's students, who was basically taking a golf cart and turning it into an autonomous shuttle.

It turned out to work pretty well, and it got people in Singapore excited, which in turn got us further excited.

2014, they did a demo where they let people of Singapore come and ride around in these carts in a garden and that worked great over the course of a weekend.

Around this time, we'd started Newtonomy, we'd actually started a commercial enterprise.

It kind of stepped at least partly away from MIT at that point.

This is a Mitsubishi IMEV electric vehicle.

When we had all of our equipment in it, the front seat was pushed forward so far that me, I'm about 6 foot 3, actually couldn't sit in the front seat so I couldn't actually accompany people on rides.

We ended up switching cars to a Renault Zoe platform, which is the 1 you see here, which had a little more legroom.

We were giving, at that point, open to the public rides in our cars in Singapore, in the part of the city that we were allowed to operate in.

As you can see, just even visually, the evolution of these systems has come a long way in a short time.

And we're just a point example of this phenomenon which is kind of, broadly speaking, similar across the industry.

But 2017 we joined Aptiv and we were excited by that because we as primarily scientists and technologists didn't have a great idea how we're gonna industrialize this technology and actually bring it to market and make it reliable and robust and make it safe, which is what I'm gonna talk about a little bit here today.

So we joined Aptiv with its global footprint.

Today we're primarily in Pittsburgh, Boston, Singapore, and Vegas.

And we've got connectivity to Aptiv's other sites in Shanghai and Wolfsburg.

Let me tell you a little bit about what's happening in Vegas.

I think people were here, when was Luke talking?

So Luke from Lyft, Luke Vincent probably talked a little bit about Vegas.

Vegas is really an interesting place for us.

We've got a big operation there, 130,000 square foot garage.

We've got 30 of those cars on the Lyft network.

So active technology, but connecting to the customer through Lyft.

So, if you go to Vegas and you open your Lyft app, it'll ask you, do you want to take a ride in an autonomous car?

You can opt in, you can opt out, it's up to you.

If you opt in, there's a reasonable chance 1 of our cars will pick you up if you call for a ride.

So, anybody can do this, competitors, innocent bystanders, totally up to you, we have nothing to hide.

Our cars are on the road 20 hours a day, 7 days a week.

If you take a ride when you get out of the car, just like any Lyft ride, you got to give us a star rating, 1 through 5.

And that to us is actually really interesting because it's a scalar, it's not too rich, but that star rating to me says something about the ride quality, meaning the comfort of the trip, the safety that you felt and the efficiency of getting to where you wanted to go.

Our star rating today is 4.95, which is pretty good.

Key numbers, we've given this point over 30,000 rides to more than 50,000 passengers.

We've driven over a million miles in Vegas and a little bit additional, but primarily there.

This one's actually in Singapore, but it's all kind of morally equivalent.

You'll see a sped up, slightly sped up view of a run from, this is now probably 6, 7 months old, on the road in Singapore, but it's got some interesting stuff in a fairly typical run.

We're on the wrong side of the road, remember because we're in Singapore.

But to give you an example of some of the types of problems we have to solve on a daily basis.

You'll see as this car is Cruising down the road, you have obstacles that we have to avoid, sometimes in the face of oncoming traffic.

We've got to deal with sometimes situations where other road users are maybe not perfectly behaving by the rules, We've got to manage that in a natural way.

Construction, Singapore like everywhere else, is pretty ubiquitous, and so you have to navigate through these less structured environments.

People who are sometimes doing things or indicating some future action which you have to make inferences about.

So typical day, a route that any 1 of us as humans would drive through without batting an eye, no problem, is actually presents some really, really complex problems for autonomous vehicles.

These are the things you have to do if you wanna be on the road, and certainly if you wanna drive millions of miles with very few accidents, which is what we're doing.

So that's an introduction to Aptiv and a little bit of background.

So let me talk about, we're gonna talk about learning and how we think about learning in the context of autonomous driving.

So there was a period a few years ago where I think as a community, people thought that we would be able to go from pixels to actuator commands with a single learned architecture, a single black box.

I'll say, generally speaking, we no longer believe that's true.

I didn't believe that was ever true, but some of us maybe thought that was true.

And I'll tell you part of the reason why, in part of this talk, a big part of it comes down to safety.

And the question of safety, convincing ourselves that that system, that black box, even if we could train it to accurately approximate this massively complex underlying function that we're trying to approximate, can we convince ourselves that it's safe?

And it's very, very hard to answer that question affirmatively.

And I'll raise some of the issues around why that is.

This is not to say that learning methods are not incredibly useful for autonomous driving because they absolutely are.

And Oscar will show you examples of why that is and how Aptiv is using some learning methods today.

But this safety dimension is tricky because there's actually 2 axes here.

1 is the actual technical safety of the system, which is to say, can we build a system that's safe, that's provably in some sense safe, that we can validate, which we can convince ourselves, achieves the intended functionality in our operational design domain that adheres to whatever regulatory requirements might be imposed on our jurisdictions that we're operating.

And there's a whole longer list related to technical safety but these are technical problems primarily.

But there's another dimension which up here is called perceived safety, which is to say when you ride in a car, even if it's safe, do you believe that it's safe?

And therefore, will you wanna take another trip?

Which sounds kinda squishy, and as engineers, we're typically uncomfortable with that kind of stuff, but it turns out to be really important and probably harder to solve because it's a little bit squishy.

And quite obviously, we gotta sit up here, right?

We gotta be in this upper right-hand corner where we have not only a very safe car from a technical perspective, but 1 that feels safe, that inspires confidence in riders, in regulators, and in everybody else.

So how do we get there in the context of elements of this system that may be black boxes, for lack of a better word?

How do we get to this point where we can trust neural networks in the context of safety critical systems, which is what an autonomous vehicle is?

It really comes down to this question of how do we convince ourselves that we can validate these systems?

Again, validating the system, ensuring that it can meet the requirements, the operational requirements in the domain of interest that are imposed by the user.

There's 3 dimensions to this key question of understanding how to validate.

And I'm gonna just briefly introduce some topics of interest around each of these.

So do we actually have confidence about what goes into this algorithm?

I mean, everybody knows garbage in, garbage out.

There's various ways that we can make this garbage.

We can have data which is insufficiently covering our domain, not representative of the domain.

We can have data that's poorly annotated by our third-party trusted partners who we've trusted to label certain things of interest.

So do we trust the data that's going in to the algorithm itself?

We've got a beautiful algorithm, super descriptive, super robust, not brittle at all, well trained, and we're running it on poor hardware, we've coded it poorly, we've got buffer overruns right and left, do we trust the implementation to actually execute in a safe manner?

Again, generally speaking, we're trying to approximate really complicated functions.

I don't think we typically use neural networks to approximate linear systems.

So this is a gnarly, nasty function which has topics of, which has problems of critical interest which are really rare.

In fact, they're the only ones of interest.

So there's these events that happen very, very infrequently that we absolutely have to get right.

It's a hard problem to convince ourselves that the algorithm is gonna perform properly in these unexpected and rare situations.

So these are the sorts of things that we think about and that we have to answer in an intelligent way to convince ourselves that we have a validated neural network based system.

Okay, let me just step through each of these topics really quickly.

So the topic of validation, what do we mean by that and why it is hard?

There's a number of different dimensions here.

The first is that we don't have insight into the nature of the function that we're trying to approximate.

You know, the underlying phenomena is really complicated.

Again, if it weren't, we'd probably, possibly be modeling it using different techniques.

We'd write a closed form equation to describe it.

Second, again, you know, the accidents, the actual crashes on the road, what's going crashes and not accidents, these are rare.

But it makes the statistical argument around these accidents and being able to avoid these accidents really, really difficult.

If you believe Rand, and they're pretty smart folks, they say you gotta drive 275 million miles without accident, without a crash, to claim a lower fatality rate than a human with 95% confidence.

Can we think about using some correlated incident, maybe some kind of close call as a proxy for accidents, which may be more frequent and maybe back in that way?

There's a lot of questions here, which I won't say we don't have any answers to, because I wouldn't go that far, but they're hard questions.

They're not questions with obvious answers.

So this is 1 of them, this issue of rare events.

The regulatory dimension is 1 of these known unknowns.

How do we validate a system if the requirements that may be imposed upon us from outside regulatory bodies are still to be written?

So there's a lack of consensus on what the safety target should be for these systems.

Smart people are thinking about this, but today it's not at all clear if you're driving in Las Vegas, if you're driving in Singapore, if you're driving in San Francisco or anywhere in between, what this target needs to be.

And then Lastly, and this is a really interesting 1, we can get through a validation process for a build of code.

But what happens when we want to update the code because obviously we will.

Does that mean we have to start that validation process again from scratch, which will unavoidably be expensive and lengthy?

But what if we only change a little bit of the code?

But what if that 1 line is like the most important line of code in the whole code base?

This is 1 that I can tell you keeps a lot of people up at night, this question of revalidation.

And then not even, again, keep that code base fixed.

And let's say that city is quite similar to your previous city, but not exactly the same.

How do we think about validation in the context of new environments?

So this continuous development issue is a challenge.

All right, let me move on to talking about the data.

There's probably people in this room who are doing active research in this area because it's a really interesting 1.

But there's a couple of obvious questions I would say that we think about when we think about data.

We can have a great algorithm and if we're training it on poor data for 1 reason or another, we won't have a great output.

So 1 thing we think about is the sufficiency, the completeness of the data and the bias that may be inherent in the data for our operational domain.

If we want to operate 24 hours a day and we only train on data collected during daytime, we're probably going to have an issue.

Annotating the data is another dimension of the problem.

We can collect raw data that's sufficient, that covers our space.

But when we annotate it, when we hand it off to a third party, because it's typically a third party, to mark up the interesting aspects of it, We provide them some specifications, but we put a lot of trust in that third party and trust that they're going to do a good job annotating the interesting parts and not the uninteresting parts, that they're going to catch all the interesting parts that we've asked them to catch, et cetera.

So this annotation part, which seems very mundane, very easy to manage, and kind of like low-hanging fruit, is in fact another key aspect of ensuring that we can trust the data.

Okay, and this reference just kind of points to the fact that there are, again, smart people thinking about this problem, which rears its head in many domains beyond autonomous driving.

Now what about the algorithms themselves?

So moving on from the data to the actual algorithm.

How do we convince ourselves that that algorithm, that like any kind of learning-based algorithm we've trained on a training set is gonna do well on some unknown test set?

Well, there's a couple kind of properties of the algorithm that we can look at, that we can kind of interrogate and kind of poke at to convince ourselves that that algorithm will perform well.

You know, 1 is invariance, and the other 1 we can say is stability.

If we make small perturbations to this function, does it behave well?

Given kind of, let's say a bounded input, do we see a bounded output Or do we see some wild response?

You know, I'm sure you've all heard of examples of adversarial images that can confuse learning-based classifiers.

You show it a turtle, it says, well, that's a turtle.

And then you show it a turtle that's maybe fuzzed with a little bit of noise that the human eye can't perceive.

So it still looks like a turtle, and it tells you it's a machine gun.

Obviously, for us in the driving domain, we want a stop sign to be correctly identified as a stop sign 100 times of 100.

We don't want that stop sign, if somebody goes up and puts a piece of duct tape in the lower right hand corner, to be interpreted as a yield sign, for example.

So this question of the properties of the algorithm, its invariance, its stability, is something of high interest.

And then lastly, to add 1 more point to this, this notion of interpretability.

So interpretability, understanding why an algorithm made a decision that it made.

This is the sort of thing that may not be a nice to have, may actually be a requirement, and would likely to be a requirement from the regulatory groups that I was referring to a minute ago.

So Let's say, imagine the case of a crash, where the system that was governing your trajectory generator was a data-driven system, was a deep learning-based trajectory generator.

Well, you may need to explain to someone exactly why that particular trajectory was generated at that particular moment.

And this may be a hard thing to do if the generator was a data-driven model.

Now, obviously, there are people working and doing active research into this specific question of interpretable learning methods, but it's a thorny 1.

It's a very, very difficult topic, and it's not at all clear to me when and if we'll get to the stage where we can, to even a technical audience, but beyond that, to a lay jury, be able to explain why algorithm X made decision Y.

Okay, so with all that in mind, let me talk a little bit about Safety.

You think, well, man, why are we taking this course with Lex because we're never gonna really use this stuff.

There's a lot of tools we can bring to bear to think about neural networks and they're generally speaking within the context of a broader safety argument.

We tend not to think about using a neural network as a holistic system to drive a car.

But we'll think about it as a sub-module that we can build other systems around.

Generally speaking, that which we can say, maybe make more rigorous claims about their performance, their underlying properties, and then therefore make a convincing, holistic safety argument that this end-to-end system is safe.

We have tools, functional safety is maybe familiar to some of you.

It's something we think about a lot in the automotive domain.

SODIF, which stands for Safety of the intended functionality.

We're basically asking ourself the question, is this overall function doing what it's intended to do?

There's kind of an analogy here to validation and verification, if you will.

And we have to answer these questions around functional safety and SODIF affirmatively, even when we have neural network based elements in order to eventually put this car on the road.

All right, so I mentioned that we need to do some embedding.

This is an example of what it might look like.

See all Lex Fridman transcripts on Youtube

Karl Iagnemma & Oscar Beijbom (Aptiv Autonomous Mobility) - MIT Self-Driving Cars