The following is a conversation with Jürgen Schmidhuber.

He's the co-director of the CS Swiss AI Lab and a co-creator of long short-term memory networks.

LSTMs are used in billions of devices today for speech recognition, translation, and much more.

Over 30 years, he has proposed a lot of interesting, out of the box ideas on meta-learning, adversarial networks, computer vision, and even a formal theory of quote, creativity, curiosity, and fun.

This conversation is part of the MIT course in Artificial General Intelligence and the Artificial Intelligence podcast.

If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D.

And now, here's my conversation with Juergen Schmidhuber.

Early on, you dreamed of AI systems that self-improve recursively.

And what was the catalyst for that birth?

What was the thing that first inspired you?

When I was a boy, I was thinking about what to do in my life and then I thought the most exciting thing is to solve the riddles of the universe and that means you have to become a physicist.

However then I realized that there is something even grander.

You can try to build a machine that isn't really a machine any longer, that learns to become a much better physicist than I could ever hope to be.

And that's how I thought maybe I can multiply my tiny little bit of creativity into infinity.

But ultimately that creativity will be multiplied to understand the universe around us.

That's the curiosity for that mystery that drove you?

Yes, so if you can build a machine that learns to solve more and more complex problems, a more and more general problem solver, then you basically have solved all the problems, at least all the solvable problems.

So how do you think, what is the mechanism for that kind of general solver look like?

Obviously we don't quite yet have 1 or know how to build 1 but we have ideas and you have had throughout your career several ideas about it.

So how do you think about that mechanism?

So in the 80s I thought about how to build this machine that learns to solve all these problems that I cannot solve myself.

And I thought it is clear it has to be a machine that not only learns to solve this problem here and this problem here, but it also has to learn to improve the learning algorithm itself.

So it has to have the learning algorithm in a representation that allows it to inspect it and modify it such that it can come up with a better learning algorithm.

So I call that meta-learning, learning to learn, and recursive self-improvement.

That is really the pinnacle of that, where you then not only learn how to improve on that problem and on that, but you also improve the way the machine improves.

And you also improve the way it improves the way it improves itself.

And that was my 1987 diploma thesis which was all about that hierarchy of meta-learners that have no computational limits except for the well-known limits that Gödel identified in

In the recent years, meta-learning has gained popularity in a specific kind of form.

You've talked about how that's not really meta-learning with neural networks, that's more basic transfer learning.

Can you talk about the difference between the big general meta-learning and a more narrow sense of meta-learning the way it's used today, the way it's talked about today?

Let's take the example of a deep neural network that has learned to classify images.

And maybe you have trained that network on 100 different databases of images.

And now a new database comes along and you want to quickly learn the new thing as well.

So 1 simple way of doing that is you take the network which already knows 100 types of databases And then you just take the top layer of that and you retrain that using the new label data that you have in the new image database.

And then it turns out that it really, really quickly can learn that too, 1 shot basically.

Because from the first 100 data sets, it already has learned so much about computer vision that it can reuse that.

And that is then almost good enough to solve the new task except you need a little bit of adjustment on the top.

So that is transfer learning and it has been done in principle for many decades.

People have done similar things for decades.

Meta-learning, true Meta-learning is about having the learning algorithm itself open to introspection by the system that is using it, and also open to modification such that the learning system has an opportunity to modify any part of the learning algorithm and then evaluate the consequences of that modification and then learn from that to create a better learning algorithm and so on recursively.

So that's a very different animal where you are opening the space of possible learning algorithms to the learning system itself.

So you've, like in the 2004 paper, you described Gator machines, programs that rewrite themselves.

Philosophically, and even in your paper mathematically, these are really compelling ideas.

But practically, do you see these self-referential programs being successful in the near term to having an impact where sort of it demonstrates to the world that this direction is a good 1 to pursue in the near term?

Yes, we had these 2 different types of fundamental research how to build a universal problem solver.

1 basically exploiting proof search and things like that that you need to come up with asymptotically optimal, theoretically optimal self-improvers and problem solvers.

However, 1 has to admit that through this proof search comes in an additive constant, an overhead, an additive overhead that vanishes in comparison to what you have to do to solve large problems.

However, for many of the small problems that we want to solve in our everyday life, We cannot ignore this constant overhead.

And that's why we also have been doing other things, non-universal things such as recurrent neural networks, which are trained by gradient descent and local search techniques, which aren't universal at all, which aren't provably optimal at all, like the other stuff that we did, but which are much more practical as long as we only want to solve the small problems that we are typically trying to solve in this environment here.

So, the universal problem solvers, like the Gödel machine, but also Markus Hutter's fastest way of solving all possible problems, which he developed around 2002 in my lab, they are associated with these constant overheads for proof search, which guarantees that the thing that you're doing is optimal.

For example, there is this fastest way of solving all problems with a computable solution which is due to Marcus, Marcus Hutter.

And to explain what's going on there let's take traveling salesman problems.

With traveling salesman problems you have a number of cities, N cities, and you try to find the shortest path through all these cities without visiting any city twice.

And Nobody knows the fastest way of solving traveling salesman problems, TSPs.

But let's assume there is a method of solving them within n to the

operations, where n is the number of cities.

Then the universal method of Marquess is going to solve the same Trolley and Salesman problem also within n to the 5 steps, plus O of 1, plus a constant number of steps that you need for the proof searcher which you need to show that this particular class of problems, the traveling salesman problems, can be solved within a certain time bound, within order n to the 5 steps, basically.

And this additive constant doesn't care for n, which means as n is getting larger and larger, as you have more and more cities, the constant overhead pales in comparison.

And that means that almost all large problems are solved in the best possible way.

Already today we already have a universal problem solver like that.

However, it's not practical because the overhead, the constant overhead is so large that for the small kinds of problems that we want to solve in this little biosphere.

By the way, when you say small, you're talking about things that fall within the constraints of our computational systems.

So they can seem quite large to us mere humans,

So they seem large and even unsolvable in a practical sense today, but they are still small compared to almost all problems because almost all problems are large problems, which are much larger than any constant.

Do you find it useful as a person who has dreamed of creating a general learning system, has worked on creating 1, has done a lot of interesting ideas there, to think about P versus NP, this formalization of how hard problems are, how they scale, this kind of worst case analysis type of thinking, do you find that useful?

Or is it only just a mathematical, It's a set of mathematical techniques to give you intuition about what's good and bad.

So P versus NP, that's super interesting from a theoretical point of view.

And in fact, as you are thinking about that problem, you can also get inspiration for better practical problem solvers.

On the other hand, we have to admit that at the moment, the best practical problem solvers for all kinds of problems that we are now solving through what is called AI at the moment, they are not of the kind that is inspired by these questions.

There we are using general purpose computers such as recurrent neural networks, But we have a search technique which is just local search, gradient descent, to try to find a program that is running on these recurrent networks such that it can solve some interesting problems, such as speech recognition or machine translation and something like that.

And there is very little theory behind the best solutions that we have at the moment that can do that.

Or can we go, Can we create a general intelligence systems without ever really proving that that system is intelligent in some kind of mathematical way?

Solving machine translation perfectly or something like that within some kind of syntactic definition of a language.

Or can we just be super impressed by the thing working extremely well and that's sufficient.

There's an old saying, and I don't know who brought it up first, which says, there's nothing more practical than a good theory.

And a good theory of problem solving under limited resources, like here in this universe or on this little planet, has to take into account these limited resources.

And so probably there is lurking a theory which is related to what we already have, these asymptotically optimal problem solvers, which tells us what we need in addition to that to come up with a practically optimal problem solver.

So I believe we will have something like that and maybe just a few little tiny twists are necessary to to change what we already have to come up with that as well.

As long as we don't have that, we admit that we are taking suboptimal ways and recurrent neural networks and long-short-term memory equipped with local search techniques, and we are happy that it works better than any competing method, but that doesn't mean that we think we are done.

You've said that an AGI system will ultimately be a simple 1.

A general intelligence system will ultimately be a simple 1.

Maybe a pseudocode of a few lines will be able to describe it.

Can you talk through your intuition behind this idea?

Why you feel that at its core, intelligence is a simple algorithm?

Experience tells us that the stuff that works best is really simple.

So the asymptotically optimal ways of solving problems, if you look at them, are just a few lines of code.

Although they have these amazing properties, just a few lines of code.

Then the most promising and most useful practical things maybe don't have this proof of optimality associated with them.

However, they are also just a few lines of code.

The most successful recurrent neural networks, you can write them down in 5 lines of pseudocode.

That's a beautiful, almost poetic idea, but what you're describing there is the lines of pseudocode are sitting on top of layers and layers of abstractions in a sense.

So you're saying at the very top it'll be a beautifully written sort of algorithm.

But do you think that there's many layers of abstractions we have to first learn to construct?

We are building on all these great abstractions that people have invented over the millennia, such as matrix multiplications and real numbers and basic arithmetics and calculus and derivations of error functions and derivatives of error functions and stuff like that.

So without that language that greatly simplifies our way of thinking about these problems, we

always, we are standing on the shoulders of the giants who in the past simplified the problem of problem solving so much that now we have a chance to do the final step.

If we take a step back through all of human civilization and just the universe in general, how do you think about evolution and what if creating a universe is required to achieve this final step?

What if going through the very painful and inefficient process of evolution is needed to come up with this set of abstractions that ultimately lead to intelligence?

Do you think there's a shortcut, or do you think we have to create something like our universe in order to create something like human level intelligence?

So far the only example we have is this 1, this universe in which we

Maybe not, But we are part of this whole process.

So apparently, so it might be the case that the code that runs the universe is really, really simple.

Everything points to that possibility because gravity and other basic forces are really simple laws that can be easily described also in just a few lines of code, basically.

And then there are these other events, the apparently random events in the history of the universe, which as far as we know at the moment don't have a compact code.

But who knows, maybe somebody in the near future is going to figure out the pseudo random generator which is which is computing whether the measurement of that spin up or down thing here is going to be positive or negative.

you ultimately think quantum mechanics is a pseudo random number, so it's all deterministic.

So a couple of years ago, a famous physicist, quantum physicist, Anton Zeilinger, he wrote an essay in nature.

1 of the fundamental insights of the 20th century was that the universe is fundamentally random on the quantum level.

And that whenever you measure spin up or down or something like that, a new bit of information enters the history of the universe.

And while I was reading that I was already typing the response and they had to publish it because I was right.

That there is no evidence, no physical evidence for that.

So there's an alternative explanation where everything that we consider random is actually pseudo-random, such as the decimal expansion of pi,

So pi is interesting because every three-digit sequence, every sequence of 3 digits, appears roughly 1 in a thousand times.

And every five-digit sequence appears roughly 1 in 10000 times.

But there's a very short algorithm, a short program that computes all of that.

And who knows, maybe tomorrow somebody, some grad student at CERN goes back over all these data points, better decay and whatever, and figures out, oh, it's the second billion digits of pi, or something like that.

We don't have any fundamental reason at the moment to believe that this is truly random and not just a deterministic video game.

If it was a deterministic video game it would be much more beautiful because beauty is simplicity And many of the basic laws of the universe, like gravity and the other basic forces, are very simple.

So very short programs can explain what these are doing.

The history of the universe would be ugly if, for the extra things, the seemingly random data points that we get all the time, that we really need a huge number of extra bits to describe all these extra bits of information.

So as long as we don't have evidence that there is no short program that computes the entire history of the entire universe, we are, as scientists, compelled to look further for that shortest program.

Your intuition says there exists a shortest, a program that can backtrack to the to the creation of the universe.

So the shortest path to the creation of the universe.

Yes, including all the entanglement things and all the spin up and down measurements that have been taken place since

So yeah, so we don't have a proof that it is random.

We don't have a proof that it is compressible to a short program.

But as long as we don't have that proof, we are obliged as scientists to keep looking for that simple explanation.

So you said simplicity is beautiful or beauty is simple.

But you also work on curiosity, discovery, the romantic notion of randomness, of serendipity, of being surprised by things that are about you, kind of in our poetic notion of reality, we think as humans require randomness.

A universe that is compressible to a short program is much more elegant and much more beautiful than another 1 which needs an almost infinite number of bits to be described.

As far as we know, Many things that are happening in this universe are really simple in terms of short programs that compute gravity and the interaction between elementary particles and so on.

So all of that seems to be very, very simple.

Every electron seems to reuse the same sub-program all the time as it is interacting with other elementary particles.

If we now require an extra oracle injecting new bits of information all the time for these extra things which are currently not understood, such as better decay, then the whole description length of the data that we can observe of the history of the universe would become much longer and therefore uglier.

Again, simplicity is elegant and beautiful.

All the history of science is a history of compression progress.

described sort of as we build up abstractions and you've talked about the idea of compression.

How do you see this, the history of science, the history of humanity, our civilization, and life on earth as some kind of path towards greater and greater compression.

Indeed, the history of science is a history of compression progress.

Hundreds of years ago, there was an astronomer whose name was Kepler and he looked at the data points that he got by watching planets move and then he had all these data points and suddenly it turned out that he can greatly compress the data by predicting it through an ellipse law.

So it turns out that all these data points are more or less on ellipses around the Sun.

And another guy came along whose name was Newton and before him Hooke.

And they said the same thing that is making these planets move like that is what makes the apples fall down.

And it also holds for stones and for all kinds of other objects.

And suddenly many many of these observations became much more compressible.

Because as long as you can predict the next thing, given what you have seen so far, you can compress it.

And then there was still something wrong with that theory of the universe and you had deviations from these predictions of the theory.

And 300 years later another guy came along whose name was Einstein.

And he was able to explain away all these deviations from the predictions of the old theory through a new theory which was called the general theory of relativity, which at first glance looks a little bit more complicated and you have to warp space and time, but you can't phrase it within 1 single sentence, which is no matter how fast you accelerate and how fast or hard you decelerate and no matter what is the gravity in your local framework, light speed always looks the same.

And from that you can calculate all the consequences.

So it's a very simple thing and it allows you to further compress all the observations because suddenly there are hardly any deviations any longer that you can measure from the predictions of this new theory.

So all of science is a history of compression progress.

You never arrive immediately at the shortest explanation of the data but you're making progress.

Whenever you are making progress you have an insight.

You see, oh first I needed so many bits of information to describe the data, to describe my falling apples, my video of falling apples.

I need so many data, so many pixels have to be stored.

But then suddenly I realized, no, there is a very simple way of predicting the third frame in the video from the first 2.

And maybe not every little detail can be predicted, but more or less most of these orange blobs that are coming down, they accelerate in the same way, which means that I can greatly compress the video.

And the amount of compression, progress, That is the depth of the insight that you have at that moment.

That's the fun that you have, the scientific fun, the fun in that discovery.

And we can build artificial systems that do the same thing, that measure the depth of their insights as they are looking at the data which is coming in through their own experiments.

And we give them a reward, an intrinsic reward, in proportion to this depth of insight.

And since they are trying to maximize the rewards they get, they are suddenly motivated to come up with new action sequences, with new experiments that have the property that the data that is coming in as a consequence of these experiments has the property that they can learn something about, see a pattern in there which they hadn't seen yet before.

So there's an idea of power play that you've described, a training in general problem solving this kind of way of looking for the unsolved problems.

Can you describe that idea a little further?

So normally what you do in computer science, You have some guy who gives you a problem, and then there is a huge search space of potential solution candidates.

And you somehow try them out, and you have more or less sophisticated ways of moving around in that search space until you finally found a solution which you consider satisfactory.

That's what most of computer science is about.

Power Play just goes 1 little step further and says, let's not only search for solutions to a given problem, but let's search for pairs of problems and their solutions, where the system itself has the opportunity to phrase its own problem.

So we are looking suddenly at pairs of problems and their solutions, or modifications of the problem solver that is supposed to generate a solution to that new problem.

And this additional degree of freedom allows us to build curious systems that are like scientists in the sense that they not only try to solve and try to find answers to existing questions, no, they are also free to pose their own questions.

So if you want to build an artificial scientist, you have to give it that freedom and power play is exactly doing that.

So that's a dimension of freedom that's important to have.

But how hard do you think that, how multi-dimensional and difficult the space of then coming up with your own questions is.

So it's 1 of the things that as human beings we consider to be the thing that makes us special, the intelligence that makes us special is that brilliant insight that can create something totally new.

Let's look at the set of all possible problems that you can formally describe, which is infinite.

Which should be the next problem that a scientist or a power play is going to solve?

Well, it should be the easiest problem that goes beyond what you already know.

So it should be the simplest problem that the current problem solver that you have, which can already solve 100 problems, that he cannot solve yet by just generalizing.

So it has to require a modification of the problem solver, such that the new problem solver can solve this new thing, but the old problem solver cannot do it.

And in addition to that, we have to make sure that the problem solver doesn't forget any of the previous solutions.

And so by definition, PowerPlay is now trying always to search in this pair of in the set of pairs of problems and problem solver modifications for a combination that minimizes the time to achieve these criteria.

So it's always trying to find the problem which is easiest to add to the repertoire.

So Just like grad students and academics and researchers can spend their whole career in a local minima, stuck trying to come up with interesting questions, but ultimately doing very little.

Do you think it's easy in this approach of looking for the simplest and solvable problem to get stuck in a local minima, is never really discovering new, you know, really jumping outside of the 100 problems that you've already solved in a genuine creative way?

No, because that's the nature of power play, that it's always trying to break its current generalization abilities by coming up with a new problem, which is beyond the current horizon.

Just shifting the horizon of knowledge a little bit out there, breaking the existing rules such that the new thing becomes solvable, but wasn't solvable by the old thing.

So like adding a new axiom, Like what Gödel did when he came up with these new sentences, new theorems that didn't have a proof in the formal system, which means you can add them to the repertoire, hoping that they are not going to damage the consistency of the whole thing.

So in the paper with the amazing title, Formal Theory of Creativity, Fun and Intrinsic Motivation, you talk about discovery as intrinsic reward.

So if you view humans as intelligent agents, what do you think is the purpose and meaning of life for us humans?

Is, you've talked about this discovery, do you see humans as an instance of power play agents?

Yeah, so humans are curious and that means they behave like scientists, not only the official scientists, but even the babies behave like scientists and they play around with their toys to figure out how the world works and how it is responding to their actions.

And that's how they learn about gravity and everything.

And yeah, in 1990, we had the first systems like that, which just try to play around with the environment and come up with situations that go beyond what they knew at that time, and then get a reward for creating these situations, and then becoming more general problem solvers, and being able to understand more of the world.

So yeah, I think in principle that curiosity strategy or more sophisticated versions of what I just described, they are what we have built in as well.

Because evolution discovered that's a good way of exploring the unknown world.

And a guy who explores the unknown world has a higher chance of solving problems that he needs to survive in this world.

On the other hand, those guys who were too curious, they were weeded out as well.

Apparently, in our society, there is a certain percentage of extremely explorative guys.

And it doesn't matter if they die, because many of the others are more conservative.

And so, yeah, it would be surprising to me if that principle of artificial curiosity wouldn't be present in almost exactly the same form here.

See all Lex Fridman transcripts on Youtube

Juergen Schmidhuber: Godel Machines, Meta-Learning, and LSTMs | Lex Fridman Podcast #11