This is a guest talk for course 6.S094: Deep Learning for Self-Driving Cars taught in Winter 2017.

Course website: https://selfdrivingcars.mit.edu
Contact: deepcars@mit.edu

Playlist: https://goo.gl/SLCb1y

Links to individual lecture videos for the course:

Lecture 1: Introduction to Deep Learning and Self-Driving Cars
https://youtu.be/1L0TKZQcUtA

Lecture 2: Deep Reinforcement Learning for Motion Planning
https://youtu.be/QDzM8r3WgBw

Lecture 3: Convolutional Neural Networks for End-to-End Learning of the Driving Task 
https://youtu.be/U1toUkZw6VI

Lecture 4: Recurrent Neural Networks for Steering through Time 
https://youtu.be/nFTQ7kHQWtc

Lecture 5: Deep Learning for Human-Centered Semi-Autonomous Vehicles 
https://youtu.be/ByZF8_-OJNI

CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- AI Podcast: https://lexfridman.com/ai/
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Twitter: https://twitter.com/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Slack: https://deep-mit-slack.herokuapp.com

He's a professor at Stanford University where he studies how to build autonomous cars that perform at or beyond human levels both on the racetrack and on public roads.

So that includes a race car that goes 120 miles an hour autonomously on the racetrack.

He spent most of 2016 as the chief innovation officer at the United States Department of Transportation and was part of the team that developed the federal automated vehicle policy.

So he deeply cares about the role that artificial intelligence plays in our society, both from the technology side and the policy perspective.

So he is now, I guess you could say a policy wonk, a world renowned engineer, and I think always a car guy, yes?

So he told me that he did a Q&A session with a group of third graders last week, and he answered all of their heart-hitting questions.

So I encourage you guys to continue on that thread and ask Chris questions after his talk.

Great, Lex, thanks for that great introduction and thanks for having me here to talk to everybody today.

So this is sort of my first week back in a civilian role.

So I'm no longer speaking and officially representing the department, although some of the slides are very similar to things that I use to speak and represent the department.

So I think as of Friday, this was still fairly current.

But I am sort of talking in my own capacity here.

So I wanted to talk about both the technology side and the policy side of automated vehicles, and in particular, how some of the techniques that you're learning in this class around deep learning and neural networks really place some challenges on regulators and policy makers attempting to ensure vehicle safety.

So just a bit about some of the cars in my background.

I am a car guy, and I've gotten a chance to work on a lot of cool ones.

I actually have been working in automated vehicles since 1992.

And the Lincoln Town Cars in the upper corner are part of an automated highway project I worked on as a PhD student at Berkeley.

I then went to Freightliner Heavy Trucks in Daimler Benz and worked with suspensions on heavy trucks before coming to Stanford and doing things like building P1 in the upper right corner there.

That's an entirely student-built electric steer-by-wire, drive-by-wire vehicle.

We've also instrumented vintage race cars, electrified a DeLorean, which I'll show a little bit later, and worked, as Lex mentioned, with Shelly, which is our self-driving Audi TT, which is an automated race car.

In addition to the Stanford work, I was a co-founder of Peloton Technology, which is a truck platooning firm, looking at bringing platooning technology, so vehicle-to-vehicle communication, which allows for shorter following distance out on the highway.

So these are some of the things I've had a chance to work with.

To give you a little bit of a sense, this is Shelly going around the racetrack at Thunder Hill.

She can actually go up to about 120 miles an hour or so on that track.

It's really just limited by the length of the straight.

It's kind of fun to watch from the outside, a little disconcerting.

Occasionally, as you see, there's nobody in the car.

Although from inside, it actually looks all pretty chill.

So Shelly, we've been working with her for a while out on the track.

She's able to get performance now, which exceeds the capability of anybody on the development team.

In fact, actually, most of my PhD students have their novice racing license.

We make sure that they get that license before going out on the track and testing.

So Shelly can beat anybody in the research group.

She actually can beat the president of the track, David Vaughn, now.

And we've had the opportunity to work recently with J.R.

Hildebrand, the IndyCar driver who finished sixth this last year in the Indy 500.

But he's actually only about a second or so faster on a minute and 25 second lap.

So we're approaching his performance and he's actually helping us get there.

Now the interesting thing about this is that we've approached this problem really from 1 of physics.

So the car is really out there calculating what it needs to do to break down into the next corner, how much grip that it thinks it has, and so forth as it's going around the track.

It's not actually a learning approach at its core, although we've added on top a number of algorithms for learning, because it turns out that the difference between the car's performance and the human performance is really getting that last little bit of capability out of the tires.

Humans drive instinctively in a way, the best humans at any rate, drive instinctively in a way which is constantly pushing to the limits of the car's capability.

And so if you sort of prejudge what those limits are, you're not going to be quite as fast.

And so that's 1 of the things we've actually been working with learning algorithms on is to try to figure out, well, how much friction do I have in this particular corner?

And how is that changing as the tires warm up and as the track warms up from the course of the morning till the afternoon?

These are the things that we need to be fast on the racetrack, but they're also the things that you need to take into account to be safe in the real world.

Because what we're trying to do with this project is understand how the car can drive at the maximum capability of the limits of the friction between the tire and the road.

Now, race car drivers do that to be fast.

As they say in racing, if you want to finish first, first you have to finish.

So it's important that they actually be fast, but also accident free.

So we're trying to learn the same things so that on the road, when you may have unknown conditions ahead of you, the car can make the safest maneuver that's using all the friction between the tire and the road to avoid ultimately any accident that the car would be physically capable of avoiding.

We've gotten to drive the car up Pikes Peak in the Bonneville Salt Flats.

Actually Shelly appeared in an Audi commercial with Zach Quinto and Leonard Nimoy.

And so at the end of the commercial, they both look at each other and declare it fascinating.

So if you're as big of a science fiction fan as I am, you realize that once your work has been declared fascinating by 2 Spocks, there's nowhere to go.

So I had to take a stint and try something different in government.

And so I spent the last year as the first chief innovation officer at the US Department of Transportation, which I think honestly was the coolest gig in the federal government.

Because I really didn't have any assigned day-to-day responsibilities, but I got to kind of dive in and help with all manner of really cool projects, including the development of the first federal automated vehicle policy.

So it was a really great opportunity to sort of see things from a different perspective.

And so what I wanted to do was, you know, kind of coming into this from an engineer, give you a perspective of what is it like from somebody looking at the regulatory side on vehicle safety, and how are they thinking about the technologies you're developing, And where does that actually leave some opportunities for engineers to make some big contributions to society?

So let's start with what vehicle safety is like today.

So today we have a system of federal motor vehicle safety standards.

They're minimum performance requirements.

And each of them must have associated with it an objective test.

So you can tell, does a vehicle meet this requirement or does it not meet this requirement?

Now interestingly, there is no federal agency that is testing vehicles before they are sold.

We rely in this country on a system of manufacturer self-certification.

So the government puts these rules out there and manufacturers go, we got this, we can meet this, and then they self-certify and put the vehicles out on the market.

The National Highway Traffic Safety Administration can then purchase vehicles and test them and make sure that they comply, but we rely on manufacturer self-certification.

This is a different system than in most of the rest of the world, which actually has pre-market certification, where before you can sell it, the government agency has to say, yes, we've checked it, and it meets all the requirements.

Aviation in this country, for instance, has that.

Aircraft require certification before they can be sold.

In 1965, Ralph Nader released a book entitled Unsafe at Any Speed.

And this is often thought of as a book about the Corvair.

The Corvair featured prominently in there as an example of a design that Nader considered to be unsafe.

What was very interesting about this book was that he was actually advocating for things like airbags and anti-lock brakes back in 1965.

These technologies didn't come along until much later.

His argument was that the auto industry had failed.

It wasn't a failure of engineering, but it was a failure of imagination.

And if you're interested in vehicle safety, I would really recommend you read this book, because It's fascinating.

They have quotes from people in the 1960s basically saying that we believe that any collision more than about 40 or 45 miles an hour is not survivable.

Therefore, there's no reason for seatbelts.

There's no reason for collapsible steering wheels.

In fact, there's a quote from somebody who made great advances in road safety saying, I can't conceive of what help a seat belt would give you beyond like firmly bracing yourself with your hands.

Those of you who have studied physics know that's kind of patently ridiculous.

But there was a common feeling that there was no sense of doing anything about vehicle crashworthiness, because once you got above a certain speed, it was inherently unsurvivable.

And I think it's interesting to look at that today, because if we were to be in a collision, I think if any of us were to be in a collision around about 40 miles an hour in a modern automobile, we'd probably expect to walk away.

We wouldn't really be thinking about our survival.

And so what this did is it led to a lot of public outcry and ultimately the National Traffic and Motor Vehicle Safety Act in 1966, which established NHTSA and established this set of federal motor vehicle safety standards.

Now the process to get a new standard made, which is a rulemaking process in government, is very time consuming.

Optimistically, about the minimum time it can possibly take is 2 years.

And so if you think about going through this process, that's really problematic.

I mean, think about what we were talking about with automated vehicles 2 years ago or 7 years ago.

And think about trying to start 7 years ago and make laws that are going to determine how those vehicles operate on the road today.

And the other thing is, is that if you think about it, our system evolved from really this sense of failure of imagination.

That the government needs to say, hey industry, do this, stop slacking off, these are the requirements, get there.

But I think It's hard to argue today with all the advances in automation that there is any failure of imagination on the part of industry.

People are coming up with all sorts of ideas and concepts for new transportation and automation.

Tech companies, startup companies, large OEMs, There's all sorts of concepts being tested out on the road.

It's hard to argue that there's still any lack of imagination.

Now, the question is, are things like this legal?

Well, from the federal level, there's an interesting report that came out about 10 months ago from the folks across the street at Volpe who did a scan and said, well, what are the things that might prevent you, based on the current federal motor vehicle safety standards, from putting an automated vehicle out on the road?

If you have a vehicle, if you start and you automate a vehicle that is currently meeting all the standards, because there are no standards that relate specifically to automation, you can certify your vehicle as meeting the federal motor vehicle safety standards.

Therefore, there's nothing at the federal level that prevents, in general, an automated vehicle from being put on the road.

So if there isn't a safety standard that you have to meet, then you can put a vehicle out on the road that meets all the existing ones and does something new, and there's no federal barrier to that.

There were a few points in there that referenced a driver, and in fact NHTSA gave an interpretation of the rule, which is 1 of the things that they can do is to say, well, we're going to give an interpretation.

It's not making a new rule, but basically interpreting the ones that we have.

And they said that actually these references to the driver could, in fact, refer to the AI system.

And so that actually is now a policy statement from the department that many of the references to driver in the Federal Motor Vehicle Safety Standards can be replaced with your self-driving AI system and the rules applied accordingly.

So in fact, there's very little that prevents you from putting a vehicle out on the road if it meets the current standards.

So if it's a modern production car, automate it.

Federal motor vehicle safety standards don't stop that.

Now, a lot of the designs that I showed, though, things that wouldn't have a steering wheel or other things are actually not compliant, because there are requirements that you have a steering wheel, that you have pedals.

Again, these are best practices that evolved in the days, of course, when people were not thinking of cars that could drive themselves.

And so these things would require an exemption by NHTSA, a process of saying that, OK, this vehicle is allowed on the road, even though it doesn't meet the current standards because it meets some equivalent.

And Setting that equivalent can be a bit of a challenge.

OK, so the question then is, well, all right, if the federal government is responsible, and it's by the Traffic Safety Act is responsible for safety on the roads, but it can't prevent people from putting anything out, what do you do?

1 approach is to say, well, let's get some federal motor vehicle safety standards out there.

But as we already said, that's probably about a seven-year process.

And if you were to start setting in best practices now, what would that look like?

We want to encourage this technology to come out onto the roads and be tested, because that's the way you're gonna learn, to get the real world data, to get the real world experience.

At the same time, the federal government is responsible for safety on the nation's roads.

So if you do put your automated system out on the highway and it's deemed to present an unreasonable risk to safety, even if you're an aftermarket manufacturer, the government can tell you to take that off the road.

But the question is, how can you do better?

How can you be proactive to try to have a discussion here.

So we know standards are maybe not the best way of doing that because they're too slow.

We'd like to make sure the public is protected, but this technology gets tested.

And so the approach taken to sort of provide some encouragement for this innovation while at the same time looking at safety, was the Federal Automated Vehicle Policy, which rolled out in September.

So this was an attempt to really say, okay, let's put out a different framework from the Federal Motor Vehicle Safety Standards.

Let's actually put out a system of voluntary guidance.

So what NHTSA is doing is to ask manufacturers to voluntarily follow certain guidance and submit to the agency a letter that they have followed a certain safety assessment.

Now, the interesting thing is that the way that this is set up is not to tell manufacturers how to do something, but really to say, these are the things that we want you to address.

And we want you to come to us to explain how you've addressed them, with the idea that from this, best practices will emerge.

We'll be able to figure out in the future what really is the best way of ensuring some of these safety items.

So you see you've got the Massachusetts license plate.

If you do put Gaudi stickers on your car, then you get closer to the center.

So that's something to consider for future reference.

But this was rolled out in Washington, DC, by the secretary.

And consists largely of multiple parts, but I think the most relevant to vehicle design is this 15-point safety assessment.

So these are the 15 points that are assessed.

And I'd like to kind of talk about a few of these in some more detail.

And it starts with this concept of an operational design domain and minimal risk or fallback conditions.

And what that means is instead of trying to put a taxonomy on here and say, well, your automation system could be an adaptive cruise control that works on the highway or it could be fully self-driving or it might be something that operates a low-speed shuttle, The guidance asks the manufacturers to define this.

And the definition is known as operational design domain.

So in other words, you tell us where your system is supposed to work.

Is it supposed to work in restricted areas?

Or is this sort of something that operates only in daylight hours in the sunshine in this area of South Florida?

But it's incumbent upon the manufacturer or developer to define the operational design domain.

And then once you've defined where the system operates, you need to define how you make sure that it is only operating in those conditions.

How do you make sure that the system stays there?

And what's your fallback in case it doesn't?

Obviously, if this is a car which is normally human driven, as you see here from the Volvo Drive Me experiment, it might be reasonable to say, we're going to ask the human driver to retake control.

Whereas, clearly, If you're going to enable blind passengers or you are going to have a vehicle that has no steering wheel, you need a different fallback system.

And so within the guidance, it really allows manufacturers to have a lot of different concepts of what they want their automation to be, so long as they can define where it works, what the fallback is in the event that it doesn't work, and how you have educated the consumer about what your technology does and what it doesn't do, so that people have a good understanding of the system performance.

A few things, if we go down, you see also validation methods and ethical considerations are aspects that are brought up here as well.

And so validation methods are really interesting as it applies to AI.

So really, the idea is that there's lots of different ways that you might test an automated vehicle.

You might go out on a test track and run it through a series of standard maneuvers.

You may develop a certain number of miles of experience driving in real world traffic and figure out how does the vehicle behave in a limited environment.

There's questions about a test track, obviously, because you don't have the sort of unknowns that can happen in the real world environment.

But if you test in 1 real-world environment, you also have a question of, is this transferable information?

So if I've driven a certain number of miles in Mountain View, California, does that tell me anything about how the vehicle is likely to behave in Cambridge, Massachusetts?

It's a little bit hard to extrapolate sometimes.

And then finally, there's also the idea of simulation and analysis.

So if I can record these situations, if I can actually create a virtual environment of the sorts of things that I see on the road, maybe I can actually run the vehicle through many, many of these scenarios, perturbed in some way, and actually test the system much more robustly in simulation than I could ever actually do out on the road.

So the guidance is actually neutral on which of these techniques manufacturers take and allow manufacturers to approach it in different ways.

And I think based upon conversations, when you think about the way companies develop this, they do take all these different approaches.

A company like Tesla, for instance, which is recording all the data streams from all their vehicles, basically, is able to run ideas or technologies silently in their vehicle.

They can actually test systems out, get real world data, and then decide whether or not to make that system active.

Companies that don't have that access to data really can't use that sort of development method and may rely much more heavily on simulation or test track experience.

So the guidance really doesn't have a particular blend of this.

And In fact, it does envision that you might have over the air software updates in the future.

So it is interesting though to think about whether you have data-driven approaches, things like artificial neural networks or whether you actually start to program in hard and fast rules.

Because as you start to think about requirements on a system, how do you actually set requirements on a system which has learned its behavior and you don't necessarily know what the internal workings or algorithms look like?

There's another 1 that comes up, which is the ethical considerations.

I'm going to pick on MIT for a moment here.

So this is an area that I actually did a lot of work on with Stanford together with some philosophers who joined our group.

And so when people hear ethical considerations in automated vehicles, it often conjures up the trolley car problem.

And so this sort of classic formulation here about the fact that you have a self-driving car which is heading towards a group of 10 people and it can either plow in and kill those 10 people or it can divert and kill the driver.

And these are classic questions in philosophy.

You actually look in fact at the trolley car problem, which is I have a runaway trolley car, and I need to either divert it to another track where it will kill somebody who's wandering across that track, or the 5 people on the trolley car are killed.

Well, in fact, as this article points out, it's like, before automated vehicles can become widespread, Car makers must solve an impossible ethical dilemma of algorithmic morality.

So if all this wasn't hard enough, I mean, you're understanding how tough the technology is to actually program this stuff.

And then you have to get the regulations right.

And now we actually have to solve impossible philosophical questions.

Well, I don't think that's actually true.

And I think, you know, it's good for engineers to work with philosophers, but not to be so literal about this.

This is a question that philosophers can ask, but engineers might ask a number of different questions like, who's responsible for the brakes on this trolley?

Why am I headed into a group of 10 people without any capability to stop?

So an engineer would, in fact, have to answer this question, but might approach it much differently.

So if I look at the trolley car problem, I might say, OK, let's see, my options are I've got a trolley car which is out of control.

First of all, I'd like to have an emergency braking system.

Well, there's a chance that that could break as well.

So if my emergency, if my base braking system goes and my emergency braking system goes, my next option would be to divert it to this side track.

Well, Knowing that that's my option, I should probably put up a fence with a warning sign that says, do not cross runaway trolley track.

Now let's say that I've done all of that.

I have to divert the trolley and somebody has ignored my sign and crossed over the fence and now is hit by the trolley.

Do I feel a little differently about this whole scenario than I did at the beginning of just trying to decide who lived and who died?

The solution was made but by thinking of it as an engineer trying to reduce risk and not by thinking of levels of morality and who deserves to live or die.

And so I think this is a very important issue, and the reason it's in the guidance is not to basically have everybody solve trolley car problems, But to try to think about these larger issues.

See all Lex Fridman transcripts on Youtube

Chris Gerdes (Stanford) on Technology, Policy and Vehicle Safety - MIT Self-Driving Cars