This is lecture 4 of course 6.S094: Deep Learning for Self-Driving Cars taught in Winter 2017.

INFO:
Slides: http://bit.ly/2Hc2zhf
Website: https://deeplearning.mit.edu
GitHub: https://github.com/lexfridman/mit-deep-learning
Playlist: https://goo.gl/SLCb1y

Links to individual lecture videos for the course:

Lecture 1: Introduction to Deep Learning and Self-Driving Cars
https://youtu.be/1L0TKZQcUtA

Lecture 2: Deep Reinforcement Learning for Motion Planning
https://youtu.be/QDzM8r3WgBw

Lecture 3: Convolutional Neural Networks for End-to-End Learning of the Driving Task 
https://youtu.be/U1toUkZw6VI

Lecture 4: Recurrent Neural Networks for Steering through Time 
https://youtu.be/nFTQ7kHQWtc

Lecture 5: Deep Learning for Human-Centered Semi-Autonomous Vehicles 
https://youtu.be/ByZF8_-OJNI

CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- AI Podcast: https://lexfridman.com/ai/
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Twitter: https://twitter.com/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Slack: https://deep-mit-slack.herokuapp.com

regular neural networks, fully connected neural networks.

We've talked about convolutional neural networks that work with images.

We've talked about reinforcement, deep reinforcement learning, where we plug in a neural network into a reinforcement learning algorithm when an agent, when a system has to not only perceive the world but also act in it and collect reward.

perhaps the least understood but the most exciting neural network out there, flavor of neural network is recurring neural networks.

The first administrative stuff, There's a website, I don't know if you heard, cars.mit.edu where you should create an account if you're a registered student.

You need to have an account if you want to get credit for this.

You need to submit code for DeepTraffic.js and DeepTesla.js.

And for DeepTraffic, you have to have a neural network that drives faster than 65 miles an hour.

If you need help to achieve that speed, please email us.

For those of you who are old school S&L fans, there's a deep thoughts section now in the profile page where we encourage you to talk about the kinds of things you tried in Deep Traffic or any of the other Deep Tesla or any of the work you've done as part of this class for deep learning.

Okay, We've talked about the vanilla neural networks on the left.

The vanilla neural network is the 1 where it's computing, it's approximating a function that maps from 1 input to 1 output.

An example is mapping images to the number that's shown in the image.

For ImageNet, it's mapping an image to what's the object in the image.

Now in fact, convolutional neural networks can operate on audio.

You could give it a chunk of audio, 5 second audio clip, that still counts as 1 input because it's fixed size.

As long as the size of the input is fixed, that's 1 chunk of input.

And as long as you have ground truth that maps that chunk of input to some output, ground truth.

That's a vanilla neural network, whether that's a fully connected neural network or a convolutional neural network.

the mysterious recurrent neural networks.

They compute functions from 1 to many, from many to 1, from many to many, also bi-directional.

sequences, time series, audio, video, whenever there's a sequence of data and that temporal dynamics that connects the data is more important than the spatial content of each individual frame.

So when there's a lot of information being conveyed in the sequence, in the temporal change of whatever that type of data is, that's when you want to use a current neural networks.

And the power of this is that for many of them of a current neural network where they really shine is when the size of the input is variable.

So you don't have a fixed chunk of data that you're putting in, it's variable input.

So you can give it a sequence of speech, several seconds of speech, and then the output is a single label of whether the speaker is male or female.

You can also do many to many translation.

You can also do many-to-many translation.

can have natural language put into the network

Machine translation, that's many-to-many.

And that many-to-many doesn't have to be mapped directly into same size sequences.

So for video, the sequence size might be the same.

And you can label every single frame, counting the number of people in every single frame.

That's many to many when the size of the input and size of the output is

The question was, are there any models where there's feedback from output and input?

And that's exactly what recurrent neural networks are.

It produces output and it copies that output and loops it back in.

That's exactly almost the definition of our current neural network.

There's a loop in there that produces the output and also takes that output as input once again.

And so There's also many to many where the sequences don't align.

the output sequence might be totally different than the input sequence.

We'll look at a lot of cool applications like you can start a song, so learn on the audio of a particular song and have the recurring neural network continue that song after a certain period of time.

So you can learn to generate sequences of audio, of natural language, of video.

I know I promised not many equations, but this is

so beautifully simple that we have to cover backpropagation.

It's also the thing that if you're a little bit lazy and you go on the internet and start using the basic tutorials for TensorFlow, you ignore how backpropagation work at your peril.

I give it some inputs, some outputs, and it's like Lego pieces.

I can assemble them like you might have done with deep traffic.

A bunch of layers, put them together and then just press train.

And backpropagation is the mechanism that neural networks currently, the best mechanism we know of that is used for training.

So we need to understand the simple power of backpropagation, but also the dangers.

Say up at the top of the slide, there's an input to the network as an image.

There's a bunch of neurons all with differentiable smooth activation functions on each neuron.

And then as you pass through those activation functions, taking the input, pass it through this net of differentiable compute nodes, you produce an output.

And that output, you also have a ground truth, the correct, the truth that you hope, you expect the network to produce.

And then you can look at the difference between what the network actually produced and what you hope to produce and that's an error.

And then you back or propagate that error, punishing or rewarding

the network that resulted in that output.

Let's start with a really simple example.

that takes as input up on top 3 variables X, Y and Z.

The function does 2 things, it adds X and Y and then it multiplies that sum by Z.

And then we can formulate that as a circuit, circuit of gates, where there's a plus gate and a multiplication gate.

And let's take some inputs shown in blue.

Let's say X is negative 2, Y is 5, Z is negative 4.

And let's do a forward pass through this circuit to produce the output.

This is so simple and so important to understand.

I just want to take my time through this because everything else about neural networks just builds on these concepts.

The output of the circuit of this network, if you think of it as such, is negative 12.

And so the forward pass is shown in blue, the backward pass will be shown in red in a second here.

So what we want to do is, what would make us happy, what would make F happy is for the output to be as high as possible.

Negative 12 is so-so, we could do better.

How do we adjust X, Y, and Z such that it produces a higher F.

Okay, let's start backward, the backward pass.

We make the gradient on the output 1, meaning we want this to increase.

that fact that we have to go up by 1, we have to look at the gradient on each 1 of the gates.

a partial derivative with respect to its inputs.

The partial derivative of the output of a gate with respect to its inputs.

If you don't know what that means, it's just how much does the output change when

If I increase X for the first function of addition, f of X, Y equals X plus Y.

If I increase X by a little bit, what happens to F?

If I increase Y by a little bit, what happens to F?

So taking a partial derivative of those with respect to X and Y, you just get a slope of 1.

So when you increase X, F increases linearly.

So the partial derivative of F with respect to X is Y, partial derivative of F with respect to Y is X.

So If you think about it, what happens is the gradients, when you change X, the gradient of change doesn't care about X.

back propagate that 1, indication of what makes us happy, backwards.

And that's done by computing the local gradient.

For Q, so the partial derivative of F with respect to Q, that intermediate value, that gradient will be negative 4.

It will take the value of Z, as I said, it's a multiplication gate.

And the same for the partial derivative of f with respect to z, it will assign that to q, the value of the forward pass from the q.

So there's a 3 and a negative 4 in the forward pass in blue and then that's flipped, negative 4 and 3 on the backward pass.

And then we continue in the same exact process.

So what makes all of this work is the chain rule.

So what it allows us to do is to compute the gradient, the gradient on F with respect to the inputs XYZ, we don't need to construct the giant function that is

the partial derivative of F with respect to X and Y and Z analytically.

We can do it step by step, backpropagating the gradients.

We can multiply the gradients together as opposed to doing partial derivative of F with respect to X.

We have just the intermediate, the local gradient of F with respect to Q and of Q with respect to X and multiply them together.

So instead of computing the gradient of that giant function x plus y equals times z.

In this case it's not that giant but it gets pretty giant in neural networks.

We look at the first function, simple addition, Q equals X plus Y and the second function multiplication F equals Q times Z.

So, the gradient on X and Y, the partial derivative of F with respect to X and Y is computed by multiplying the gradient on the output,

negative 4, times the gradient on the inputs, which as we talked about when the operation is addition, that's just 1.

The gradient of the partial derivative of F with respect to XYZ.

That means, so for X and Y it's negative 4, for Z it's 3.

That means in order to make F happy, we have to decrease the inputs that have a negative gradient and increase the inputs that have

The negative ones are X and Y, the positive is Z.

Hopefully I don't say the word beautiful too many times in this presentation, but this is very simple, beautifully simple.

It has no knowledge of the broader happiness of F.

It just propagates, it computes the gradient between the output and the input.

And you can propagate this gradient based on, in

this case, F, a gradient of 1, but also just the error.

Instead of 1, we could have on the output the error is the measure of happiness.

And then we could propagate that error backwards.

These gates are important because you can break down almost every operation we could think of that we work with in neural networks into 1 of several gates like this.

And the most popular are 3, which is addition, multiplication and the max operation.

So for addition, what you do is you ignore the...

So, okay, the process is you take a forward pass to the network.

and through the backward pass you compute those gradients.

For an add gate, you equally distribute the gradients on the output to the input.

So when the gradient on the output is negative 4, you equally distribute it to negative 4.

So that 3 is ignored when you back propagate it.

On the multiplication gate, on the multiply gate, it's trickier.

If you look at F, that's a multiply gate.

The forward pass values are switched and multiplied by the value of the gradient in the

If it's confusing, go through the slides slowly.

1 more gate, there's the max gate, which takes the inputs and produces as output the value that is larger.

And when computing the gradient of the max gate,

It distributes the gradient similar to the add gate but to only 1.

So it, unlike the add gate, pays attention to the input values on the forward pass.

the whole point here is it's like, it's really simple.

What a neural network is just a simple collection of these gates.

And you take a forward pass, you calculate some kind of function on the end, a gradient at

the very end, and you propagate that back.

So usually for neural networks, that's an error function, a loss function, objective function, cost function, all the same word.

When you have 3 weights, w0, w1, w2, and x, 2 inputs, x0, x1, that's going to

But then you could decompose that neuron, you could separate it all into just a set of gates like this.

Addition, multiplication, there's exponential in there and division.

there's 5 inputs, there's 3 weights and 2 inputs, X0, X1.

You take a forward pass through this circuit.

In this case again, you want it to increase so that the gradient on the output is 1.

You back propagate that gradient of 1 to the inputs.

Now, in neural networks, there's a bunch of parameters that you're trying to, through this process, modify.

You get to modify the weights along the way and the biases.

The inputs are fixed, the outputs are fixed, the outputs that you hope the network will produce.

So I get to try to adjust those weights in such that in the direction of the gradient.

The main way that neural networks learn is we update the weights and the biases to decrease the loss function.

on top left, the simple network, 3 inputs, 3 weights on each of

the inputs, there's a bias on the node B, it produces an output A.

And loss is computed as y minus a squared divided by 2

where y is the ground truth, the output that you want the network to produce.

And that loss function is back propagated in exactly the same way that we described before.

Alright, so the subtasks that are involved in this update of weights and biases is that the forward pass computes the network output at every neuron and then finally the output layer, computes the error, the difference between A and B, and then backward propagates the gradients.

Instead of 1 on the output, it will be the error on the output and you back propagate it.

And then once you know the gradient, you adjust the weights and the biases in the direction of the gradient, or actually the opposite of the direction of the gradient because you want the loss to decrease.

And the amount by which you make that adjustment is called the learning rate.

The learning rate could be the same across the entire network or it could be individual to every weight.

And the process of adjusting the weights and biases is just optimization.

You have an objective function and you're trying to minimize it.

And your variables are the parameters, the weights and biases.

And neural networks just happen to have tens, hundreds of thousands, millions of those parameters.

So the space, the function that you're trying to minimize is highly nonlinear.

You have 2 weights here, 2 plots, or actually 1 weight, sorry, 1 weight.

And then as you adjust it, the cost, you adjust in such a way that minimizes the output cost.

And there's a bunch of optimization methods for doing this.

This is a convex function, so you can find the minimum, the local minimum, if you know about these kind of terminologies, the local minimum is the same as the global minimum.

So it's not a weirdly hilly terrain where you can get stuck in.

So your goal is to get to the bottom of this thing and if it's really complex terrain, it'll be hard to get to the bottom of it.

So there is a lot of different, this general approach is gradient descent.

And there's a lot of different ways to do gradient descent.

Some adding, in various ways of adding randomness into the process.

So you don't get stuck into the weird crevices of the terrain.

This is the part you have to be aware of.

When you're designing a network for deep traffic and nothing is happening, this might be what's happening.

Vanishing gradients or exploding gradients.

When the partial derivative is small, so if you take the sigmoid function, the most popular

for a while activation function, the derivative is 0 at the tails.

So when the input to the sigmoid function is really high or really low, that derivative is going to be 0.

So the gradient that you compute, you know, gradient tells you how much I want to adjust the weights.

And so you back propagate that 0, a very low number, and it gets less and less as you back propagate, and so the result is that you don't, you think that you don't need to adjust the weights at all, and when a large fraction of the network thinks that weights don't need to be adjusted, then they don't adjust the weights and you're not doing any learning.

There's a piecewise, the Rayleigh function,

which is the most popular activation function.

if the neurons are initialized poorly, this function might be 0 gradient for the entire dataset.

Nothing that you produce as input, You run all your thousands of images of cats and none of them fire at all.

So you have to pick these, both the optimization

engine, the solver that you use, and the activation functions carefully.

You can't just plug and play like they're Legos.

is the vanilla optimization algorithm for gradient descent, for optimizing the loss function over the gradients.

And so what's visualized here is, again, if you've done any numerical optimization, nonlinear optimization, there's the famous saddle point that's tricky for these algorithms to deal with.

What happens is it's easy for them to oscillate, get stuck in that saddle and oscillate back and forth.

As opposed to what they want to do which is go down into, you know, you get so happy that you found this like low point, that you forget that there's a much lower point.

And so you get stuck with the gradient, the momentum of the gradient keeps rocking back and forth without you going to a much greater global minimum.

And there's a bunch of clever ways of solving that.

But in this case, as long as the gradients don't vanish,

SGD, the stochastic gradient descent, 1 of these algorithms will get you there.

That might take a little while, but it'll get you there.

See all Lex Fridman transcripts on Youtube

MIT 6.S094: Recurrent Neural Networks for Steering Through Time