First deep learning intro lecture of course 6.S094: Deep Learning for Self-Driving Cars taught in Winter 2017.

INFO:
Slides: http://bit.ly/2HmL5ia
Website: https://deeplearning.mit.edu
GitHub: https://github.com/lexfridman/mit-deep-learning
Playlist: https://goo.gl/SLCb1y

Links to individual lecture videos for the course:

Lecture 1: Introduction to Deep Learning and Self-Driving Cars
https://youtu.be/1L0TKZQcUtA

Lecture 2: Deep Reinforcement Learning for Motion Planning
https://youtu.be/QDzM8r3WgBw

Lecture 3: Convolutional Neural Networks for End-to-End Learning of the Driving Task 
https://youtu.be/U1toUkZw6VI

Lecture 4: Recurrent Neural Networks for Steering through Time 
https://youtu.be/nFTQ7kHQWtc

Lecture 5: Deep Learning for Human-Centered Semi-Autonomous Vehicles 
https://youtu.be/ByZF8_-OJNI

CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- AI Podcast: https://lexfridman.com/ai/
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Twitter: https://twitter.com/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Slack: https://deep-mit-slack.herokuapp.com

So welcome to course 6S094, Deep Learning for Self-Driving Cars.

deep learning, of deep neural networks using the guiding case study of building self-driving cars.

You get to listen to me for majority of these lectures and I am part of an amazing team with some brilliant TAs, would you say?

Spencer, William Angell, Spencer Dodd, and all the way in the back, the smartest and the tallest person I know, Benedict Jenick.

So what you see there on the left of the slide is a visualization of 1 of the 2 projects that, 1 of the 2 simulations games that we'll get to go through.

as a way to teach you about deep reinforcement learning but also as

a way to excite you by challenging you to compete against others if you wish to win a special prize yet to be announced.

So you can reach me and the TAs at deepcars.mit.edu

if you have any questions about the tutorials, about the lecture, about anything at all.

has the lecture content, code tutorials again.

Like today, the lecture slides for today are already up in PDF form.

The slides themselves, if you want to see them, just email me, but they're over a gigabyte in size because they're very heavy in videos so I'm just posting the PDFs.

And there will be lecture videos available a few days after the lecture is given.

So speaking of which, there is a camera in the back.

This is being videotaped and recorded, but for the most part, the camera is just on the speaker.

If that kind of thing worries you, then you could sit on the periphery of the classroom or maybe I suggest sunglasses and a fake mustache, that would be a good idea.

There is a competition for the game that you see on the left.

In order to get credit for the course, you have to design a neural network that drives the car just above the speed limit of 65 miles an hour.

But if you want to win, you need to go a little faster than that.

You may be new to programming, new to machine learning, new to robotics or you're an expert in those fields but want to go back to the basics.

So what you will learn is an overview of deep reinforcement learning, a convolutional neural networks, recurrent neural networks, and how these methods can help improve each of the components of autonomous driving, perception, visual perception, localization, mapping, control planning and the detection of driver state.

There is, in this particular formulation of it, there is 7 lanes.

It looks like a game, but I assure you it's very serious.

The car in red is being controlled by a neural network and we'll explain how

you can control and design the various aspects, the various parameters of this neural network.

So this we're using ConvNetJS, which is a library that is programmed by Andrej Karpathy in JavaScript.

So amazingly we live in a world where you can train in a matter of minutes a neural network in your browser.

The reason we did this is so that there is very few requirements to get you up and started with neural networks.

So in order to complete this project for the course, you don't need any requirements except to have a Chrome browser and to win the competition you don't need anything except a Chrome browser.

The second project, codenamed Deep Tesla or Tesla,

is using data from a Tesla vehicle of the forward roadway and using end-to-end learning, taking the image and putting it into convolutional neural network that directly maps a regressor that maps to a steering angle.

So all it takes is a single image and it predicts a steering angle for the car.

And we have data for the car itself and you get to build a neural network that tries

to do better, tries to steer better or at least as good as the car.

Okay, let's get started with the question,

with the thing that we understand so poorly at this time because it's so shrouded in mystery but it fascinates many of us And that's the question of what is intelligence?

This is from a 1996, March 1996 Time Magazine.

And the question, can machines think is answered below with they already do.

So what if anything is special about the human mind?

It's a good question for 1996, a good question for 2016, 17 now and the future.

Can an artificial intelligence system achieve a well-defined, specifically, formally defined finite set of goals?

And this little diagram from a book that got me into artificial intelligence as a bright-eyed high school student, the Artificial Intelligence, a Modern approach.

It has a set of sensors that do the perception.

It takes those sensors in, does something magical, there's a question mark there, and with a set of effectors, acts in the world, manipulates objects in that world.

And so, special purpose, we can, under this formulation, as long as the environment is formally defined, well-defined, as long as a set of goals are well-defined, as long as a set of actions, sensors, and the ways that the perception carries itself out as well-defined, We have good algorithms of which we'll talk about that can optimize for those goals.

The question is, if we inch along this path, Will we get closer to the general formulation, to the general purpose version of what artificial intelligence is?

Can it achieve poorly defined, unconstrained set of goals with an unconstrained, poorly defined set of actions and unconstrained, poorly defined utility functions, rewards.

This is what we do pretty well most days.

Exist in an undefined, full of uncertainty world.

So, okay, we can separate tasks into 3 different categories.

It didn't seem so at the birth of artificial intelligence.

That's in fact true if you think about it.

Playing board games, theorem proving, all the kind of mathematical logic problems that can be formally defined.

So this is where a lot of the exciting breakthroughs have been happening where machine learning methods, data-driven methods can help aid or improve on the performance of our human experts.

This means medical diagnosis, hardware design, scheduling.

And then there is the thing that we take for granted, the trivial thing, the thing that we do so easily every day when we wake up in the morning, the mundane tasks of everyday speech, of written language, of visual perception, of walking, which we'll talk about in today's lecture, is a fascinatingly difficult task and object manipulation.

So the question is that we're asking here, before we talk about deep learning, before we talk about the specific methods, we really want to dig in and try to see what is it about driving.

Is it more like chess which you see on the left there, where we can formally define a set of lanes, a set of actions and formulate it as this, you know, there's 5 set of actions.

You can change a lane, you can avoid obstacles, you can formally define an obstacle, you can formally define the rules of the

road or is there something about natural language, something similar to everyday conversation about driving that requires a much higher degree of reasoning, of communication, of learning, of existing in this under actuated space.

Is it a lot more than just left lane, right lane, speed up, slow down.

What are the sensors we get to work with on an autonomous vehicle?

And we'll get a lot more in depth on this, especially with the guest speakers who built many of these.

There's radar, there's the range sensors, radar, lidar that give you information about the obstacles in the environment.

That help localize the obstacles in the environment.

There's the visible light camera and stereo vision that gives you texture information that helps you figure out not just where the obstacles are but what they are, helps to classify those, helps to understand their subtle movements.

Then there is the information about the vehicle itself, about the trajectory and the movement of the vehicle that comes from the GPS and IMU sensors.

And there is the state of the rich state of

What are all the individual systems doing?

And there is 1 of the less studied but fascinating to us on the research side is audio.

The sounds of the road that provide the rich context of a wet road, the sound of a road that when it stopped raining but it's still wet, the sound that it makes, The screeching tire and honking, these are all fascinating signals as well.

And the focus of the research in our group, the thing that's really much under investigated is the internal facing sensors.

The driver, sensing the state of the driver.

That comes from the visual information and the audio information.

If you were to break into modules, the task of what it means to build a self-driving vehicle.

First, you want to know where you are, where am I, localization and mapping.

You want to map the external environment, figure out where all the different obstacles are, all the entities are, and use that estimate of the environment to then figure out where I am, where the robot is.

It's understanding not just the positional aspects of the external environment and the dynamics of it, but also what those entities are.

Once you have kind of figured out to the best of your abilities, your position and the position of other entities in this world, that's figuring out a trajectory through that world.

And finally, once you've figured out how to move about safely and effectively through that world, it's figuring out what the human that's on board is doing.

Because as I will talk about the path to a self-driving vehicle and that is hence our focus on Tesla, may go through semi-autonomous vehicles where the vehicle must not only drive itself but effectively hand over control from the car to the human and back.

Well, there's a lot of fun stuff from the 80s and 90s but

the big breakthroughs came in the second DARPA Grand Challenge with Stanford Stanley when they won the competition, 1 of 5 cars that finished.

a fully autonomous vehicle was able to complete the race in record time.

the task was no longer a race to the desert but through a urban environment.

And a lot of that work led directly into the acceptance and large major industry players taking on the challenge of building these vehicles.

Tesla with its Autopilot system and now Autopilot 2 system.

And there's many other companies including 1 of the speakers for this course of Neutronomy that are driving the wonderful streets of Boston.

We have, if we think about the accomplishments in the DARPA challenge and if we look at the accomplishments of the Google self-driving car which essentially boils the world down into a chess game.

It uses incredibly accurate sensors to build a three-dimensional map of the world, localize itself effective in that world and move about that world

Now what if driving, the Open question is, if driving is more like a conversation, like a natural language conversation, how hard is it to pass the Turing test?

The Turing test, as the popular current formulation is, can a computer be mistaken for a human being in more than 30% of the time.

When a human is talking behind a veil, having a conversation with either

a computer or a human, they mistake the other side of that conversation for being a human when it's in fact a computer.

And the way you would in natural language build a system that has successfully passes the Turing test is the natural language processing part to enable it to communicate successfully.

So generate language and interpret language.

Then you represent knowledge, the state of the conversation, transferred over time.

And the last piece, and this is the hard piece, is the automated reasoning.

Can we teach machine learning methods to reason?

That is something that will propagate through our discussion because

as I will talk about the various methods, the various deep learning methods, neural networks are good at learning from data but there's no good mechanism for reasoning.

Now, reasoning could be just something that we tell ourselves we do to feel special, better to feel like we're better than machines.

Reasoning may be simply something as simple as learning from data.

We just need a larger network or there could be a totally different mechanism required and we'll talk about the possibilities there.

No, it's very difficult to find these kind of situations in the United States.

So the question was for this video, is it in the United States or not?

are much more towards the direction of natural language versus chess.

In the United States, generally speaking, we follow rules more concretely.

The quality of roads is better, the marking on the roads is better, so there's less requirements there.

Yeah, so, but it's certainly not the United States.

I'm pretty, I'm, I spent quite a bit of Googling trying to find the United States and it's difficult.

So let's talk about the recent breakthroughs

in machine learning and what is at the core of those breakthroughs is neural networks that have been around for a long time and I will talk about what has changed, what are the cool new things and what hasn't changed and what are its possibilities.

I know there's a few folks here, neuroscience folks.

And so the human neuron has inspired the artificial neuron, the computational building block of a neural network, of an artificial neural network.

These neurons for both artificial and human brains are interconnected.

In the human brain there's about, I believe, 10,000 outgoing connections from every neuron

on average and they're interconnected to each other.

The largest current, as far as I'm aware, artificial neural network has 10 billion of those connections, synapses.

Our human brain, to the best estimate that I'm aware of, has 10,000 times that.

It takes a set of inputs, it puts a weight on each of those inputs, sums them together, applies a bias value on each that sits on each neuron and using an activation function that takes as input that sum plus the bias and squishes it together to produce a 0 to 1 signal.

to take a few inputs and produces an output, a classification for example, a 0,1.

And as we'll talk about simply, it can serve as a linear classifier.

So it can draw a line, it can learn to draw a line between, like what's seen here, between the blue dots and the yellow dots.

And that's exactly what we'll do in the IPython notebook that I'll talk about.

But the basic algorithm is you initialize the weights on the inputs

You perform this previous operation I talked about, sum up, and compute the output.

And if the output does not match the ground truth, the expected output, the output that it should produce, the weights are punished accordingly.

And we'll talk through a little bit of the math of that.

And this process is repeated until the Perceptron does not make any more mistakes.

Now here's the amazing thing about neural networks.

1 on the mathematical side is the universality of neural networks.

With just a single layer, if we stack them together, a single hidden layer, the inputs on the left, the outputs on the right, and in the middle there's a single hidden layer.

you can think of, that you know, you can think of driving as a function.

It takes as input the world outside as output, the control of the vehicle.

There exists a neural network out there that can drive perfectly.

So we can think of this then, these functions as a special purpose function, special purpose intelligence.

You can take, say as input, the number of bedrooms, the square feet, the type

It passes that value through to the hidden layer and then 1 more step, it produces the final price estimate for the house or for the residence.

And we can teach a network to do this pretty well in a supervised way.

You provide a lot of examples where you know the number of bedrooms, the square feet, the type of neighborhood, and then you also know the final price of the house or the residence.

And then you can, as I'll talk about through a process of backpropagation, teach these networks

Now, some of the exciting breakthroughs recently have been in the general purpose intelligence.

This is from Andrei Karpathy, who is now at OpenAI.

I would like to take a moment here to try to explain how amazing this is.

there's 2 paddles and you're trying to bounce the ball back and in such a way that prevents the other guy from bouncing the ball back at you.

The artificial intelligence agent is on the right in green and up top is the score, 8 to 1.

Now this takes about 3 days to train on a regular computer, this network.

They're slightly processed and also you take the difference between 2 frames

but it's basically the raw pixel information.

and the output is a single probability of moving up.

and what it's doing is it learns Not, you don't know at any 1 moment, you don't know what the right thing to do is.

You only know what the right thing to do is by the fact that eventually you win or lose the game.

There's no supervised learning about, there's no like universal fact about any 1 state being good or bad and any 1 action being good or bad in any state.

But if you punish or reward every single action you took, every single action you took for entire game based on the result.

So no matter what you did, if you won the game, the end justifies the means.

If you won the game, every action you took and every action state pair gets rewarded.

And this process with only 200,000 games, where the system just simulates the games, It can learn to beat the computer.

This system knows nothing about Pong, nothing about games.

This is general intelligence, except for the fact that it's just a game of Pong.

talk about how this can be extended further, why this is so promising

and why this is also, we should proceed with caution.

So again, there's a set of actions you take, up, down, up, down, based on the output of the network.

Given the probability of moving up, you move up or down based on the output of the network.

And you have a set of states and every single state action pair is awarded if there's a win and it's punished if there's a loss.

When you go home, think about how amazing that is.

And if you don't understand why that's amazing,

The question was, what is supervised learning, what is unsupervised learning, what's the difference?

So supervised learning is, when people talk about machine learning, they mean supervised learning most of the time.

Supervised learning is learning from data, it's learning from example.

When you have a set of inputs and a set of outputs that you know are correct, what are called ground truth.

So you need those examples, a large amount of them, to train any of the machine learning algorithms to learn to then generalize that to future examples.

This is, actually there's a third 1 called reinforcement learning where the ground truth is sparse.

The information about when something is good or not, the ground truth, only happens every once in a while, at the end of the game, not every single frame.

And unsupervised learning is when you have no information about the outputs that are correct or incorrect.

And it is the excitement of the deep learning community, it's unsupervised learning, but it has achieved no major breakthroughs at this point.

This is the, I'll talk about what the future of deep learning is and a lot of the people that are working in the field are excited by it.

But right now, any interesting accomplishment has to do with supervised learning.

Can you talk about, so, I guess the green 1, right?

And the brown 1 is just a heuristic solution, like, look at the velocity.

So, basically the reinforcement learning here is learning from somebody who has certain rules and how can that be guaranteed that you would generalize to somebody else?

So the question was, this, the green paddle learns to play this game successfully

against this specific 1 brown paddle with operating under specific kinds of rules.

How do we know it can generalize to other games, other things?

But the mechanism by which it learns generalizes.

So as long as you let it play in whatever world you want it to succeed in long enough, it will use the same approach to learn to succeed in that world.

The problem is this works for worlds you can simulate well.

Unfortunately, 1 of the big challenges of neural networks is that they're not currently efficient learners.

Human beings need 1 example oftentimes and they learn very efficiently from that 1 example.

So if you think about the way a human being would approach this game, this game of Pong, they would only need a simple set of instructions.

You're in control of a paddle and you can move it up and down and your task is to bounce the ball past the other player controlled by AI.

Now, a human being would immediately, they may not win the game, but they would immediately understand the game and will be able to successfully play it well enough to pretty quickly learn to beat the game.

See all Lex Fridman transcripts on Youtube

MIT 6.S094: Introduction to Deep Learning and Self-Driving Cars