This is a clip from a conversation with Francois Chollet on the Artificial Intelligence podcast from Sep 2019. You can watch the full conversation here: http://bit.ly/2O28EyF or listen to it here: https://lexfridman.com/francois-chollet/ and subscribe on your podcast app. You can watch other clips on the clips playlist: http://bit.ly/2JYkbfZ or full episodes on the full episode playlist: http://bit.ly/2EcbaKf Please note, I select clips with insights from these much longer conversation with the hope of helping make ideas from the podcast more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a clip every day when there is not a full length video. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Full episode: http://bit.ly/2O28EyF
Full episodes playlist: http://bit.ly/2EcbaKf
Clips playlist: http://bit.ly/2JYkbfZ
Podcast website: https://lexfridman.com/ai

François Chollet is the creator of Keras, which is an open source deep learning library that is designed to enable fast, user-friendly experimentation with deep neural networks. It serves as an interface to several deep learning libraries, most popular of which is TensorFlow, and it was integrated into TensorFlow main codebase a while back. Aside from creating an exceptionally useful and popular library, François is also a world-class AI researcher and software engineer at Google, and is definitely an outspoken, if not controversial, personality in the AI world, especially in the realm of ideas around the future of artificial intelligence.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

Like what does it mean to be more or less intelligent?

Is it completely coupled to a particular problem?

Or is there something a little bit more universal?

Yeah, I do believe all intelligence is specialized intelligence.

Even human intelligence has some degree of generality.

Well, all intelligence systems have some degree of generality but they're always specialized in 1 category of problems.

So the human intelligence is specialized in the human experience.

And that shows at various levels that shows in some prior knowledge, that's innate that we have at birth.

Knowledge about things like agents, goal-driven behavior, visual priors about what makes an object, priors about time, and so on.

That shows also in the way we learn, for instance, it's very, very easy for us to pick up language.

It's very, very easy for us to learn certain things because we are basically hard coded to learn them.

And we are specialized in solving certain kinds of problems.

And we are quite useless when it comes to other kinds of problems.

For instance, we are not really designed to handle very long-term problems.

We have no capability of seeing the very long-term.

We don't have very much working memory, you know?

Do you think long-term planning, we're talking about scale of years, millennia, what do you mean by long-term we're not very good?

Well, human intelligence is specialized in the human experience.

Even within 1 lifetime, we have a very hard time envisioning, you know, things on a scale of years, like it's very difficult to project yourself at a scale of 5 years, a scale of 10 years, and so on.

We can solve only fairly narrowly scoped problems.

So when it comes to solving bigger problems, larger scale problems, we are not actually doing it on an individual level.

We, we have this thing called civilization, right?

Which is itself a sort of problem solving system, a sort of artificial intelligence system, right?

And it's not running on 1 brain, it's running on a network of brains.

In fact, it's running on much more than a network of brains.

It's running on a lot of infrastructure, like books and computers and the internet and human institutions and so on.

And that is capable of handling problems on a much greater scale than any individual human.

If you look at computer science, for instance, that's an institution that solves problems and it is superhuman, right?

It can solve much bigger problems than an individual human could.

And science itself, science as a system, as an institution is a kind of artificially intelligent problem solving algorithm that is superhuman.

It's at least computer science is like a theorem prover.

At a scale of thousands, maybe hundreds of thousands of human beings.

At that scale, what do you think is an intelligent agent?

So there's us humans at the individual level.

There is millions, maybe billions of bacteria in our skin.

You can even go to the particle level as systems that behave, you can say intelligently in some ways, and then you can look at the Earth as a single organism, you can look at our galaxy, and even the universe as a single organism.

How do you think about scale and defining intelligent systems?

And we're here at Google, there is millions of devices doing computation in a distributed way.

do you think about intelligence versus scale?

You can always characterize anything as a system.

I think people who talk about things like intelligence explosion tend to focus on 1 agent is basically 1 brain, like 1 brain considered in isolation, like a brain, a jar that's controlling a body in a very like top to bottom kind of fashion.

And that body is pursuing goals into an environment.

You have the brain at the top of the pyramid, then you have the body just plainly receiving orders, and then the body is manipulating objects in an environment and so on.

So everything is subordinate to this 1 thing, this epicenter, which is the brain.

But in real life, intelligent agents don't really work like this, right?

There is no strong delimitation between the brain and the body to start with.

You have to look not just at the brain, but at the nervous system.

But then the nervous system and the body are not really 2 separate entities.

So you have to look at an entire animal as 1 agent.

But then you start realizing as you observe an animal over any length of time, that a lot of the intelligence of an animal is actually externalized.

A lot of our intelligence is externalized.

When you write down some notes, that is externalized intelligence.

When you write a computer program, you are externalizing cognition.

So it's externalized in books, it's externalized in computers, the internet, in other humans.

So it's, there is no like hard delimitation of what makes an intelligent agent.

Okay, but AlphaGo is better at Go than the best human player.

So do you think there's such a ability as such a concept as a intelligence explosion, a specific task?

And then, well, yeah, do you think it's possible to have a category of tasks on which you do have something like an exponential growth of ability to solve that particular problem?

I think if you consider a specific vertical, it's probably possible to some extent.

I also don't think we have to speculate about it because we have real world examples of recursively self-improving intelligent systems, right?

So for instance, science is a problem solving system, a knowledge generation system, like a system that experiences the world in some sense, and then gradually understands it and can act on it.

And that system is superhuman, and it is clearly recursively self-improving because science feeds into technology.

Technology can be used to build better tools, better computers, better instrumentation and so on, which in turn can make science faster.

So science is probably the closest thing we have today to a recursively self-improving superhuman AI.

And you can just observe, you know, is science, is scientific progress today exploding, which you know, itself is an interesting question.

And you can use that as a basis to try to understand what will happen with a superhuman AI that has science-like behavior.


Speaker 0: Can you try to define intelligence? Like what does it mean to be more or less intelligent? Is it completely coupled to a particular problem? Or is there something a little bit more universal?

Speaker 1: Yeah, I do believe all intelligence is specialized intelligence. Even human intelligence has some degree of generality. Well, all intelligence systems have some degree of generality but they're always specialized in 1 category of problems. So the human intelligence is specialized in the human experience. And that shows at various levels that shows in some prior knowledge, that's innate that we have at birth.

Knowledge about things like agents, goal-driven behavior, visual priors about what makes an object, priors about time, and so on. That shows also in the way we learn, for instance, it's very, very easy for us to pick up language. It's very, very easy for us to learn certain things because we are basically hard coded to learn them. And we are specialized in solving certain kinds of problems. And we are quite useless when it comes to other kinds of problems.

For instance, we are not really designed to handle very long-term problems. We have no capability of seeing the very long-term. We don't have very much working memory, you know?

Speaker 0: So how do you think about long-term? Do you think long-term planning, we're talking about scale of years, millennia, what do you mean by long-term we're not very good?

Speaker 1: Well, human intelligence is specialized in the human experience. And human experience is very short. Like 1 lifetime is short. Even within 1 lifetime, we have a very hard time envisioning, you know, things on a scale of years, like it's very difficult to project yourself at a scale of 5 years, a scale of 10 years, and so on.

Speaker 0: Right.

Speaker 1: We can solve only fairly narrowly scoped problems. So when it comes to solving bigger problems, larger scale problems, we are not actually doing it on an individual level. So it's not actually our brain doing it. We, we have this thing called civilization, right? Which is itself a sort of problem solving system, a sort of artificial intelligence system, right?

And it's not running on 1 brain, it's running on a network of brains. In fact, it's running on much more than a network of brains. It's running on a lot of infrastructure, like books and computers and the internet and human institutions and so on. And that is capable of handling problems on a much greater scale than any individual human. If you look at computer science, for instance, that's an institution that solves problems and it is superhuman, right?

It operates on a greater scale. It can solve much bigger problems than an individual human could. And science itself, science as a system, as an institution is a kind of artificially intelligent problem solving algorithm that is superhuman.

Speaker 0: Yeah. It's at least computer science is like a theorem prover. At a scale of thousands, maybe hundreds of thousands of human beings. At that scale, what do you think is an intelligent agent? So there's us humans at the individual level.

There is millions, maybe billions of bacteria in our skin. There is, that's at the smaller scale. You can even go to the particle level as systems that behave, you can say intelligently in some ways, and then you can look at the Earth as a single organism, you can look at our galaxy, and even the universe as a single organism. How do you think about scale and defining intelligent systems? And we're here at Google, there is millions of devices doing computation in a distributed way.

Speaker 1: How

Speaker 0: do you think about intelligence versus scale?

Speaker 1: You can always characterize anything as a system. I think people who talk about things like intelligence explosion tend to focus on 1 agent is basically 1 brain, like 1 brain considered in isolation, like a brain, a jar that's controlling a body in a very like top to bottom kind of fashion. And that body is pursuing goals into an environment. So it's a very hierarchical view. You have the brain at the top of the pyramid, then you have the body just plainly receiving orders, and then the body is manipulating objects in an environment and so on.

So everything is subordinate to this 1 thing, this epicenter, which is the brain. But in real life, intelligent agents don't really work like this, right? There is no strong delimitation between the brain and the body to start with. You have to look not just at the brain, but at the nervous system. But then the nervous system and the body are not really 2 separate entities.

So you have to look at an entire animal as 1 agent. But then you start realizing as you observe an animal over any length of time, that a lot of the intelligence of an animal is actually externalized. That's especially true for humans. A lot of our intelligence is externalized. When you write down some notes, that is externalized intelligence.

When you write a computer program, you are externalizing cognition. So it's externalized in books, it's externalized in computers, the internet, in other humans. It's externalized in language and so on. So it's, there is no like hard delimitation of what makes an intelligent agent. It's all about context.

Speaker 0: Okay, but AlphaGo is better at Go than the best human player. You know, there's levels of skill here. So do you think there's such a ability as such a concept as a intelligence explosion, a specific task? And then, well, yeah, do you think it's possible to have a category of tasks on which you do have something like an exponential growth of ability to solve that particular problem?

Speaker 1: I think if you consider a specific vertical, it's probably possible to some extent. I also don't think we have to speculate about it because we have real world examples of recursively self-improving intelligent systems, right? So for instance, science is a problem solving system, a knowledge generation system, like a system that experiences the world in some sense, and then gradually understands it and can act on it. And that system is superhuman, and it is clearly recursively self-improving because science feeds into technology. Technology can be used to build better tools, better computers, better instrumentation and so on, which in turn can make science faster.

Right? So science is probably the closest thing we have today to a recursively self-improving superhuman AI. And you can just observe, you know, is science, is scientific progress today exploding, which you know, itself is an interesting question. And you can use that as a basis to try to understand what will happen with a superhuman AI that has science-like behavior.

See all Lex Fridman transcripts on Youtube

What is Intelligence? - François Chollet and Lex Fridman | AI Podcast Clips