This is a clip from a conversation with Yann LeCun from Aug 2019. New full episodes every Mon & Thu and 2-5 new clips every Tue & Fri. You can watch the full conversation here: https://www.youtube.com/watch?v=SGSOCuByo24 or listen to it here: https://lexfridman.com/yann-lecun/ and subscribe on your podcast app. You can watch other clips on the clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41 or full episodes on the full episode playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a few clips every Tue & Fri. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Yann LeCun is one of the fathers of deep learning, the recent revolution in AI that has captivated the world with the possibility of what machines can learn from data. He is a professor at New York University, a Vice President & Chief AI Scientist at Facebook, co-recipient of the Turing Award for his work on deep learning. He is probably best known as the founder of convolutional neural networks, in particular their early application to optical character recognition.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

♪♪ You've written advice saying, don't get fooled by people who claim to have a solution to artificial general intelligence, who claim to have an AI system that works just like the human brain, or who claim to have figured out how the brain works.

Ask them what the error rate they get on MNIST or ImageNet.

But I think your opinion is still, and this is an image net, yes, maybe data, there may be new benchmarks, right?

But I think that philosophy is 1 you still and somewhat hold that benchmarks and the practical testing, the practical application is where you really get to test the ideas.

Well, it may not be completely practical.

Like for example, you know, it could be a toy dataset, but it has to be some sort of task that the community as a whole has accepted as some sort of standard, you know, kind of benchmark, if you want.

So for example, many years ago here at FAIR, people, you know, Jason West on Antoine Born and a few others proposed the Babi tasks, which were kind of a toy problem to test the ability of machines to reason actually to access working memory and things like this.

And it was very useful, even though it wasn't a real task.

It's just that I was really struck by the fact that a lot of people, particularly a lot of people with money to invest, would be fooled by people telling them, oh, we have the algorithm of the cortex and you should give us 50 million.

So There's a lot of people who try to take advantage of the hype for business reasons and so on.

But let me sort of talk to this idea that the new ideas, the ideas that push the field forward may not yet have a benchmark.

Or it may be very difficult to establish a benchmark.

Establishing benchmarks is part of the process.

So what are your thoughts about, so we have these benchmarks on, around stuff we can do with images, from classification to captioning to just every kind of information you can pull off from images in the surface level.

There's audio data sets, there's some video.

What can we start, natural language, what kind of benchmarks do you see that start creeping on to more something like intelligence, like reasoning, like maybe you don't like the term, but AGI, echoes of that kind of formulation?

A lot of people are working on interactive environments in which you can train and test intelligent systems.

So there, for example, the classical paradigm of supervised learning is that you have a dataset, you partition it into a training set, validation set, test set, and there's a clear protocol, right?

But what if that assumes that the samples are statistically independent?

You can exchange them, the order in which you see them shouldn't matter, you know, things like that.

But what if the answer you give determines the next sample you see, which is the case, for example, in robotics, right?

You robot does something and then it gets exposed to a new room and depending on where it goes, the room would be different.

So that's the, that creates the exploration problem.

The, what if the samples, so that creates also a dependency between samples, right?

You, you, if you move, if you can only move in, in space, the next sample you're going to see is going to be probably in the same building, most likely.

So all the assumptions about the validity of this training set, test set hypothesis break whenever a machine can take an action that has an influence in the world.

So people are setting up artificial environments where that takes place, right?

The robot runs around a 3D model of a house and can interact with objects and things like this.

So you do robotics by simulation, you have those, you know, opening a gym type thing, or MuJoCo kind of simulated robots, and you have games, you know, things like that.

So that's where the field is going, really, this kind of environment.

Like, I don't like the term AGI, because it implies that human intelligence is general, and human intelligence is nothing like general.

We like to think of ourselves as having general intelligence, we don't.

I think what's impressive about humans is ability to learn, as we were talking about learning, to learn in just so many different domains.

It's perhaps not arbitrarily general, but just you can learn in many domains and integrate that knowledge somehow.

It's not an example, It's more like a quasi-mathematical demonstration.

So you have about 1000000 fibers coming out of 1 of your eyes.

It's 1000000 nerve fibers, your optical nerve.

Let's imagine that they are binary, so they can be active or inactive.

So the input to your visual cortex is 1000000 bits.

Now they're connected to your brain in a particular way, and your brain has connections that are kind of a little bit like a convolutional net, they're kind of local, you know, in space and things like this.

I cut your optical nerve and I put a device that makes a random perturbation of a permutation of all the nerve fibers.

So now what comes to your brain is a fixed but random permutation of all the pixels.

There's no way in hell that your visual cortex, even if I do this to you in infancy, will actually learn vision to the same level of quality that you can.

And you're saying there's no way you've relearned that.

No, because now 2 pixels that are nearby in the world will end up in very different places in your visual cortex.

And your neurons there have no connections with each other because they only connected locally.

So this whole, our entire, the hardware is built in many ways to support.

Yeah, but it's still pretty damn impressive.

So let's imagine you want to train your visual system to recognize particular patterns of those 1000000 bits.

It's a two-way classification with 1000000 binary inputs.

How many such Boolean functions are there?

You have 2 to the 1 million combinations of inputs.

For each of those, you have an output bit.

And so you have 2 to the 1 million Boolean functions of this type, okay, which is an unimaginably large number.

How many of those functions can actually be computed by your visual cortex?

And the answer is a tiny, tiny, tiny, tiny, tiny, tiny sliver, like an enormously tiny sliver.

But, okay, that's an argument against the word general.

I think there's a, I agree with your intuition, but I'm not sure, it seems the brain is impressively capable of adjusting to things.

It's because we can't imagine tasks that are outside of our comprehension.

So we think we are general because we are general of all the things that we can apprehend.

But there is a huge world out there of things that we have no idea.

So, at least physicists call that heat, or they call it entropy, which is you have a thing full of gas, right?

It has, you know, pressure, it has a temperature has, you know, and you can write equations, PV equal NRT, you know, things like that, right?

When you reduce the volume, the temperature goes up, the pressure goes up, you know, things like that, right?

Those are the things you can know about that system.

And it's a tiny, tiny number of bits compared to the complete information of the state of the entire system, because the state of the entire system will give you the position and momentum of every molecule of the gas.

And what you don't know about it is the entropy and you interpret it as heat.

The energy contained in that thing is what we call heat.

Now, it's very possible that in fact, there is some very strong structure in how those molecules are moving.

It's just that they are in a way that we are just not wired to perceive.

And there's an infinite amount of things we're not wired to perceive.

And you're right, that's a nice way to put it.

We're general to all the things we can imagine, which is a very tiny subset of all things that are possible.

So it's like Kolmogorov complexity or the Kolmogorov-Chaitin summa of complexity.

You know, every bit string or every integer is random, except for all the ones that you can actually write down.

So beautifully put, but you know, so we can just call it artificial intelligence, we don't need to have a general.

You know, you'll start, any time you touch human, it gets interesting because, you know, it's because we attach ourselves to human and it's difficult to define what human intelligence is.

Nevertheless, my definition is maybe damn impressive intelligence, okay?

Dem impressive demonstration of intelligence, whatever.


Speaker 0: ♪♪ You've written advice saying, don't get fooled by people who claim to have a solution to artificial general intelligence, who claim to have an AI system that works just like the human brain, or who claim to have figured out how the brain works. Ask them what the error rate they get on MNIST or ImageNet.

Speaker 1: So- This is a little dated, by the way.

Speaker 0: 2000, I mean, 5 years. Yes. Who's counting? Okay. But I think your opinion is still, and this is an image net, yes, maybe data, there may be new benchmarks, right?

But I think that philosophy is 1 you still and somewhat hold that benchmarks and the practical testing, the practical application is where you really get to test the ideas.

Speaker 1: Well, it may not be completely practical. Like for example, you know, it could be a toy dataset, but it has to be some sort of task that the community as a whole has accepted as some sort of standard, you know, kind of benchmark, if you want. It doesn't need to be real. So for example, many years ago here at FAIR, people, you know, Jason West on Antoine Born and a few others proposed the Babi tasks, which were kind of a toy problem to test the ability of machines to reason actually to access working memory and things like this. And it was very useful, even though it wasn't a real task.

MNIST is kind of halfway a real task. So toy problems can be very useful. It's just that I was really struck by the fact that a lot of people, particularly a lot of people with money to invest, would be fooled by people telling them, oh, we have the algorithm of the cortex and you should give us 50 million.

Speaker 0: Yes, absolutely. So There's a lot of people who try to take advantage of the hype for business reasons and so on. But let me sort of talk to this idea that the new ideas, the ideas that push the field forward may not yet have a benchmark. Or it may be very difficult to establish a benchmark.

Speaker 1: I agree, that's part of the process. Establishing benchmarks is part of the process.

Speaker 0: So what are your thoughts about, so we have these benchmarks on, around stuff we can do with images, from classification to captioning to just every kind of information you can pull off from images in the surface level. There's audio data sets, there's some video. What can we start, natural language, what kind of benchmarks do you see that start creeping on to more something like intelligence, like reasoning, like maybe you don't like the term, but AGI, echoes of that kind of formulation?

Speaker 1: A lot of people are working on interactive environments in which you can train and test intelligent systems. So there, for example, the classical paradigm of supervised learning is that you have a dataset, you partition it into a training set, validation set, test set, and there's a clear protocol, right? But what if that assumes that the samples are statistically independent? You can exchange them, the order in which you see them shouldn't matter, you know, things like that. But what if the answer you give determines the next sample you see, which is the case, for example, in robotics, right?

You robot does something and then it gets exposed to a new room and depending on where it goes, the room would be different. So that's the, that creates the exploration problem. The, what if the samples, so that creates also a dependency between samples, right? You, you, if you move, if you can only move in, in space, the next sample you're going to see is going to be probably in the same building, most likely. So all the assumptions about the validity of this training set, test set hypothesis break whenever a machine can take an action that has an influence in the world.

And it's what it's going to see. So people are setting up artificial environments where that takes place, right? The robot runs around a 3D model of a house and can interact with objects and things like this. So you do robotics by simulation, you have those, you know, opening a gym type thing, or MuJoCo kind of simulated robots, and you have games, you know, things like that. So that's where the field is going, really, this kind of environment.

Now, back to the question of AGI. Like, I don't like the term AGI, because it implies that human intelligence is general, and human intelligence is nothing like general. It's very, very specialized. We think it's general. We like to think of ourselves as having general intelligence, we don't.

We're very specialized. We're only slightly more general than.

Speaker 0: Why does it feel general? So you kind of, the term general. I think what's impressive about humans is ability to learn, as we were talking about learning, to learn in just so many different domains. It's perhaps not arbitrarily general, but just you can learn in many domains and integrate that knowledge somehow. The knowledge persists.

Speaker 1: So let me take a very specific example. It's not an example, It's more like a quasi-mathematical demonstration. So you have about 1000000 fibers coming out of 1 of your eyes. Okay, 2000000 total. But let's talk about just 1 of them.

It's 1000000 nerve fibers, your optical nerve. Let's imagine that they are binary, so they can be active or inactive. So the input to your visual cortex is 1000000 bits. Now they're connected to your brain in a particular way, and your brain has connections that are kind of a little bit like a convolutional net, they're kind of local, you know, in space and things like this. Now imagine I play a trick on you.

It's a pretty nasty trick, I admit. I cut your optical nerve and I put a device that makes a random perturbation of a permutation of all the nerve fibers. So now what comes to your brain is a fixed but random permutation of all the pixels. There's no way in hell that your visual cortex, even if I do this to you in infancy, will actually learn vision to the same level of quality that you can.

Speaker 0: Got it. And you're saying there's no way you've relearned that.

Speaker 1: No, because now 2 pixels that are nearby in the world will end up in very different places in your visual cortex. And your neurons there have no connections with each other because they only connected locally.

Speaker 0: So this whole, our entire, the hardware is built in many ways to support.

Speaker 1: The locality of the real world.

Speaker 0: Yeah. Yes, that's specialization. Yeah, but it's still pretty damn impressive. So it's not perfect generalization. It's not even close.

Speaker 1: No, no. It's not that it's not even close. It's not at all.

Speaker 0: Yeah, it's not. It's specialized.

Speaker 1: So how many Boolean functions? So let's imagine you want to train your visual system to recognize particular patterns of those 1000000 bits. So that's a Boolean function, right? Either the pattern is here or not here. It's a two-way classification with 1000000 binary inputs.

How many such Boolean functions are there? Okay. You have 2 to the 1 million combinations of inputs. For each of those, you have an output bit. And so you have 2 to the 1 million Boolean functions of this type, okay, which is an unimaginably large number.

How many of those functions can actually be computed by your visual cortex? And the answer is a tiny, tiny, tiny, tiny, tiny, tiny sliver, like an enormously tiny sliver.

Speaker 0: Yeah, yeah.

Speaker 1: So we are ridiculously specialized. Okay.

Speaker 0: But, okay, that's an argument against the word general. I think there's a, I agree with your intuition, but I'm not sure, it seems the brain is impressively capable of adjusting to things.

Speaker 1: It's because we can't imagine tasks that are outside of our comprehension. So we think we are general because we are general of all the things that we can apprehend. But there is a huge world out there of things that we have no idea. We call that heat, by the way. Heat.

Heat. So, at least physicists call that heat, or they call it entropy, which is you have a thing full of gas, right?

Speaker 0: Closed system for gas, right?

Speaker 1: Close or no close. It has, you know, pressure, it has a temperature has, you know, and you can write equations, PV equal NRT, you know, things like that, right? When you reduce the volume, the temperature goes up, the pressure goes up, you know, things like that, right? For perfect gas, at least. Those are the things you can know about that system.

And it's a tiny, tiny number of bits compared to the complete information of the state of the entire system, because the state of the entire system will give you the position and momentum of every molecule of the gas. And what you don't know about it is the entropy and you interpret it as heat. The energy contained in that thing is what we call heat. Now, it's very possible that in fact, there is some very strong structure in how those molecules are moving. It's just that they are in a way that we are just not wired to perceive.

Speaker 0: Yeah, we're ignorant to it. And there's an infinite amount of things we're not wired to perceive. And you're right, that's a nice way to put it. We're general to all the things we can imagine, which is a very tiny subset of all things that are possible.

Speaker 1: So it's like Kolmogorov complexity or the Kolmogorov-Chaitin summa of complexity. You know, every bit string or every integer is random, except for all the ones that you can actually write down.

Speaker 0: Yeah. Okay. So beautifully put, but you know, so we can just call it artificial intelligence, we don't need to have a general.

Speaker 1: Step. Or human level, human level intelligence

Speaker 0: is good. You know, you'll start, any time you touch human, it gets interesting because, you know, it's because we attach ourselves to human and it's difficult to define what human intelligence is. Nevertheless, my definition is maybe damn impressive intelligence, okay? Dem impressive demonstration of intelligence, whatever. You

See all Lex Fridman transcripts on Youtube

Yann LeCun: Benchmarks for Human-Level Intelligence | AI Podcast Clips