This is a clip from a conversation with Francois Chollet from Sep 2019. New full episodes every Mon & Thu and 1-2 new clips or a new non-podcast video on all other days. You can watch the full conversation here: https://www.youtube.com/watch?v=Bo8MY4JpiXE
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on iTunes:
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a few clips every Tue & Fri. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

François Chollet is the creator of Keras, which is an open source deep learning library that is designed to enable fast, user-friendly experimentation with deep neural networks. It serves as an interface to several deep learning libraries, most popular of which is TensorFlow, and it was integrated into TensorFlow main codebase a while back. Aside from creating an exceptionally useful and popular library, François is also a world-class AI researcher and software engineer at Google, and is definitely an outspoken, if not controversial, personality in the AI world, especially in the realm of ideas around the future of artificial intelligence.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

What is your intuition why an intelligence explosion is not possible?

Like taking the scientific, all the scientific revolutions, why can't we slightly accelerate that process?

So you can absolutely accelerate any problem solving process.

So recursive self-improvement is absolutely a real thing.

But what happens with a recursively self-improving system is typically not explosion because no system exists in isolation.

And so tweaking 1 part of the system means that suddenly another part of the system becomes a bottleneck.

And if you look at science, for instance, which is clearly a recursively self-improving, clearly a problem solving system, Scientific progress is not actually exploding.

If you look at science, what you see is the picture of a system that is consuming an exponentially increasing amount of resources, but it's having a linear output in terms of scientific progress.

And maybe that will seem like a very strong claim.

Many people are actually saying that, you know, scientific progress is exponential, but when they're claiming this, they're actually looking at indicators of resource consumptions, resource consumption by science.

For instance, the number of papers being published, the number of patents being filed and so on, which are just, just completely correlated with how many people are working on, on, on science today.

So it's actually an indicator of resource consumption, but what you should look at is the output is, progress in terms of the knowledge that science generates in terms of the, the scope and significance of the problems that we solve.

And some people have actually been trying to measure that.

Like Michael Nielsen, for instance, he had a very nice paper, I think that was last year about it.

So his approach to measure scientific progress was to look at the timeline of scientific discoveries over the past, you know, 100, 150 years.

And for each major discovery, ask a panel of experts to rate the significance of the discovery.

And if the output of science as an institution were exponential, you would expect the temporal density of significance to go up exponentially, maybe because there's a faster rate of discoveries, maybe because the discoveries are, you know, increasingly more important.

And what actually happens if you, if you plot this temporal density of significance measured in this way is that you see very much a flat graph.

You see a flat graph across all disciplines, across physics, biology, medicine, and so on.

And it actually makes a lot of sense if you think about it, because think about the progress of physics,

Think about the progress of technology, you know, 170 years ago, when we started having, you know, replacing horses with cars, when we started having electricity and so on.

And today is also a time of very, very fast change, but it would be an unfair characterization to say that today technology and science are moving way faster than they did 50 years ago, 100 years ago.

And if you do try to rigorously plot the temporal density of the significance, yeah, of significance idea, of significance, sorry, you do see very flat curves.

And you can check out the paper that Michael Nielsen had about this idea.

And so the way I interpret it is as you make progress in a given field or in a given subfield of science, it becomes exponentially more difficult to make further progress.

Like the very first person to work on information theory.

If you enter a new field and it's still the very early years, there's a lot of low hanging fruit you can pick.

But the next generation of researchers is going to have to dig much harder, actually, to make smaller discoveries, probably larger number of smaller discoveries.

And to achieve the same amount of impact, you're going to need a much greater head count.

And that's exactly the picture you're seeing with science is that the number of scientists and engineers is in fact increasing exponentially.

The amount of computational resources that are available to science is increasing exponentially and so on.

So the resource consumption of science is exponential, but the output in terms of progress, in terms of significance is linear.

And the reason why is because, and even though science is recursively self-improving, meaning that scientific progress turns into technological progress, which in turn helps science.

If you look at computers, for instance, our products of science and computers are tremendously useful in speeding up science.

The internet, same thing, the internet is a technology that's made possible by very recent scientific advances and itself, because it enables, you know, scientists to network, to communicate, to exchange papers and ideas much faster.

It is a way to speed up scientific progress.

So even though you're looking at a recursively self-improving system, it is consuming exponentially more resources to produce the same amount of problem solving very much.

And certainly that holds for the deep learning community.

If you look at the temporal, what did you call it?

The temporal density of significant ideas.

If you look at in deep learning, I think I'd have to think about that.

But if you really look at significant ideas in deep learning, they might even be decreasing.

But the amount of papers is still today exponentially increasing.

So I think if you look at an aggregate, my guess is that you would see a linear progress.

If you were to sum the significance of all papers, you would see roughly in your progress.

And in my opinion, it is not a coincidence that you're seeing linear progress in science despite exponential resource consumption.

I think the resource consumption is dynamically adjusting itself to maintain linear progress because we as a community expect linear progress, meaning that if we start investing less and seeing less progress, It means that suddenly there are some lower hanging fruits that become available and someone's going to step up and pick them.

So it's very much like a market for discoveries and ideas.

But there's another fundamental part which you're highlighting, which is a hypothesis that science, or like the space of ideas, any 1 path you travel down, it gets exponentially more difficult to get new, to develop new ideas.

And your sense is that's gonna hold across our mysterious universe.

Yes, well, exponential progress triggers exponential friction.

So that if you tweak 1 part of the system, suddenly some other part becomes a bottleneck.

For instance, let's say we develop some device that measures its own acceleration, and then it has some engine and it outputs even more acceleration in proportion of its own acceleration, and you drop it somewhere, it's not going to reach infinite speed because it exists in a certain context.

So the air around it is going to generate friction and it's going to block it at some top speed.

And even if you were to consider the broader context and lift the bottleneck there, like the bottleneck of friction, then some other part of the system would start stepping in and creating exponential friction, maybe the speed of light or, you know, whatever.

And this definitely holds true when you look at the problem solving algorithm that is being run by science as an institution, science as a system, as you make more and more progress, despite having this recursive self-improvement component, you are encountering exponential friction.

Like the more researchers you have working on, on, on different ideas, the more overhead you have in terms of communication across researchers.

If you look at, you were mentioning quantum mechanics, right?

Well, if he wants to start making significant discoveries today, significant progress in quantum mechanics, there is an amount of knowledge you have to ingest, which is huge.

So there's the very large overhead to even start to contribute.

There's a large amount of overheads to synchronize across researchers and so on.

And of course, the significant practical experiments are going to require exponentially expensive equipment because the easier ones have already been run, right?

So in your senses, there's no way escaping, there's no way of escaping this kind of friction with artificial intelligence systems.

Yeah, no, I think science is a very good way to model what would happen with a superhuman or a classification of improving AI.

It's not like a mathematical proof of anything.

Like I'm not, I'm not trying to prove anything.

I'm just trying to make an argument to question the narrative of intelligence explosion, which is quite a dominant narrative.

And you do get a lot of pushback if you go against it because so for many people, right.

AI is not just a subfield of computer science.

It's more like a belief system, like this belief that the world is headed towards an event, the singularity, past which, you know, AI will become, will go exponential very much and the world will be transformed and humans will become obsolete.

And if you, if you go against this narrative, because, because it is not really a scientific argument, but more of a belief system.

It is part of the identity of many people.

If you go against this narrative, it's like you're attacking the identity of people who believe in it.

It's almost like saying God doesn't exist or something.


Speaker 0: What is your intuition why an intelligence explosion is not possible? Like taking the scientific, all the scientific revolutions, why can't we slightly accelerate that process?

Speaker 1: So you can absolutely accelerate any problem solving process. So recursive self-improvement is absolutely a real thing. But what happens with a recursively self-improving system is typically not explosion because no system exists in isolation. And so tweaking 1 part of the system means that suddenly another part of the system becomes a bottleneck. And if you look at science, for instance, which is clearly a recursively self-improving, clearly a problem solving system, Scientific progress is not actually exploding.

If you look at science, what you see is the picture of a system that is consuming an exponentially increasing amount of resources, but it's having a linear output in terms of scientific progress. And maybe that will seem like a very strong claim. Many people are actually saying that, you know, scientific progress is exponential, but when they're claiming this, they're actually looking at indicators of resource consumptions, resource consumption by science. For instance, the number of papers being published, the number of patents being filed and so on, which are just, just completely correlated with how many people are working on, on, on science today. Right.

So it's actually an indicator of resource consumption, but what you should look at is the output is, progress in terms of the knowledge that science generates in terms of the, the scope and significance of the problems that we solve. And some people have actually been trying to measure that. Like Michael Nielsen, for instance, he had a very nice paper, I think that was last year about it. So his approach to measure scientific progress was to look at the timeline of scientific discoveries over the past, you know, 100, 150 years. And for each major discovery, ask a panel of experts to rate the significance of the discovery.

And if the output of science as an institution were exponential, you would expect the temporal density of significance to go up exponentially, maybe because there's a faster rate of discoveries, maybe because the discoveries are, you know, increasingly more important. And what actually happens if you, if you plot this temporal density of significance measured in this way is that you see very much a flat graph. You see a flat graph across all disciplines, across physics, biology, medicine, and so on. And it actually makes a lot of sense if you think about it, because think about the progress of physics,

Speaker 0: 110

Speaker 1: years ago, right? It was a time of crazy change. Think about the progress of technology, you know, 170 years ago, when we started having, you know, replacing horses with cars, when we started having electricity and so on. It was a time of incredible change. And today is also a time of very, very fast change, but it would be an unfair characterization to say that today technology and science are moving way faster than they did 50 years ago, 100 years ago.

And if you do try to rigorously plot the temporal density of the significance, yeah, of significance idea, of significance, sorry, you do see very flat curves. That's fascinating. And you can check out the paper that Michael Nielsen had about this idea. And so the way I interpret it is as you make progress in a given field or in a given subfield of science, it becomes exponentially more difficult to make further progress. Like the very first person to work on information theory.

If you enter a new field and it's still the very early years, there's a lot of low hanging fruit you can pick.

Speaker 0: That's right. Yeah.

Speaker 1: But the next generation of researchers is going to have to dig much harder, actually, to make smaller discoveries, probably larger number of smaller discoveries. And to achieve the same amount of impact, you're going to need a much greater head count. And that's exactly the picture you're seeing with science is that the number of scientists and engineers is in fact increasing exponentially. The amount of computational resources that are available to science is increasing exponentially and so on. So the resource consumption of science is exponential, but the output in terms of progress, in terms of significance is linear.

And the reason why is because, and even though science is recursively self-improving, meaning that scientific progress turns into technological progress, which in turn helps science. If you look at computers, for instance, our products of science and computers are tremendously useful in speeding up science. The internet, same thing, the internet is a technology that's made possible by very recent scientific advances and itself, because it enables, you know, scientists to network, to communicate, to exchange papers and ideas much faster. It is a way to speed up scientific progress. So even though you're looking at a recursively self-improving system, it is consuming exponentially more resources to produce the same amount of problem solving very much.

Speaker 0: So that's a fascinating way to paint it. And certainly that holds for the deep learning community. Right? If you look at the temporal, what did you call it? The temporal density of significant ideas.

If you look at in deep learning, I think I'd have to think about that. But if you really look at significant ideas in deep learning, they might even be decreasing. So I do believe the

Speaker 1: per paper significance is decreasing. But the amount of papers is still today exponentially increasing. So I think if you look at an aggregate, my guess is that you would see a linear progress. If you were to sum the significance of all papers, you would see roughly in your progress. And in my opinion, it is not a coincidence that you're seeing linear progress in science despite exponential resource consumption.

I think the resource consumption is dynamically adjusting itself to maintain linear progress because we as a community expect linear progress, meaning that if we start investing less and seeing less progress, It means that suddenly there are some lower hanging fruits that become available and someone's going to step up and pick them. So it's very much like a market for discoveries and ideas.

Speaker 0: But there's another fundamental part which you're highlighting, which is a hypothesis that science, or like the space of ideas, any 1 path you travel down, it gets exponentially more difficult to get new, to develop new ideas. And your sense is that's gonna hold across our mysterious universe.

Speaker 1: Yes, well, exponential progress triggers exponential friction. So that if you tweak 1 part of the system, suddenly some other part becomes a bottleneck. For instance, let's say we develop some device that measures its own acceleration, and then it has some engine and it outputs even more acceleration in proportion of its own acceleration, and you drop it somewhere, it's not going to reach infinite speed because it exists in a certain context. So the air around it is going to generate friction and it's going to block it at some top speed. And even if you were to consider the broader context and lift the bottleneck there, like the bottleneck of friction, then some other part of the system would start stepping in and creating exponential friction, maybe the speed of light or, you know, whatever.

And this definitely holds true when you look at the problem solving algorithm that is being run by science as an institution, science as a system, as you make more and more progress, despite having this recursive self-improvement component, you are encountering exponential friction. Like the more researchers you have working on, on, on different ideas, the more overhead you have in terms of communication across researchers. If you look at, you were mentioning quantum mechanics, right? Well, if he wants to start making significant discoveries today, significant progress in quantum mechanics, there is an amount of knowledge you have to ingest, which is huge. So there's the very large overhead to even start to contribute.

There's a large amount of overheads to synchronize across researchers and so on. And of course, the significant practical experiments are going to require exponentially expensive equipment because the easier ones have already been run, right?

Speaker 0: So in your senses, there's no way escaping, there's no way of escaping this kind of friction with artificial intelligence systems.

Speaker 1: Yeah, no, I think science is a very good way to model what would happen with a superhuman or a classification of improving AI.

Speaker 0: That's your sense, I mean,

Speaker 1: the- That's my intuition. It's not like a mathematical proof of anything. That's not my point. Like I'm not, I'm not trying to prove anything. I'm just trying to make an argument to question the narrative of intelligence explosion, which is quite a dominant narrative.

And you do get a lot of pushback if you go against it because so for many people, right. AI is not just a subfield of computer science. It's more like a belief system, like this belief that the world is headed towards an event, the singularity, past which, you know, AI will become, will go exponential very much and the world will be transformed and humans will become obsolete. And if you, if you go against this narrative, because, because it is not really a scientific argument, but more of a belief system. It is part of the identity of many people.

If you go against this narrative, it's like you're attacking the identity of people who believe in it. It's almost like saying God doesn't exist or something. So you do get a lot of pushback if

See all Lex Fridman transcripts on Youtube

François Chollet: Scientific Progress is Not Exponential | AI Podcast Clips