The following is a conversation with Jan Lekun.

He's considered to be 1 of the fathers of deep learning, which if you've been hiding under a rock, is the recent revolution in AI that's captivated the world with the possibility of what machines can learn from data.

He's a professor at New York University, a vice president and chief AI scientist at Facebook and co-recipient of the Turing Award for his work on deep learning.

He's probably best known as the founding father of convolutional neural networks, in particular their application to optical character recognition and the famed MNIST dataset.

He is also an outspoken personality, unafraid to speak his mind in a distinctive French accent and explore provocative ideas, both in the rigorous medium of academic research and the somewhat less rigorous medium of Twitter and Facebook.

This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube, give it 5 stars on iTunes, support it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D-M-A-N.

And now, here's my conversation with Jan Lekun.

You said that 2001 Space Odyssey is 1 of your favorite movies.

Hal 9000 decides to get rid of the astronauts for people who haven't seen the movie, spoiler alert, because he, it, she believes that the astronauts, they will interfere with the mission.

Do you see Hal as flawed in some fundamental way or even evil?

There's no notion of evil in that context, other than the fact that people die.

But it was an example of what people call value misalignment, right?

You give an objective to a machine and the machine strives to achieve this objective.

And if you don't put any constraints on this objective, like don't kill people and don't do things like this, the machine, given the power, will do stupid things just to achieve this objective, or damaging things to achieve its objective.

It's a little bit like, I mean, we're used to this in the context of human society.

We put in place laws to prevent people from doing bad things because spontaneously they would do those bad things.

So we have to shape their cost function, their objective function if you want through laws to kind of correct and education obviously to sort of correct for those.

So maybe just pushing a little further on that point, Hal, you know, there's a mission, there's a, there's fuzziness around the ambiguity around what the actual mission is.

But, you know, do you think that there will be a time from a utilitarian perspective where an AI system, where it is not misalignment, where it is alignment for the greater good of society, that an AI system will make decisions that are difficult?

I mean, eventually we'll have to figure out how to do this.

And again, we're not starting from scratch because we've been doing this with humans for millennia.

So designing objective functions for people is something that we know how to do, and we don't do it by programming things, although the legal code is called code.

And it's actually the design of an objective function.

It tells you, here's what you can do, here's what you can't do.

So there is this idea somehow that it's a new thing for people to try to design objective functions that are aligned with the common good.

But no, we've been writing laws for millennia and that's exactly what it is.

So that's where the science of lawmaking and computer science will

So there's nothing special about HAL or AI systems, it's just the continuation of tools used to make some of these difficult ethical judgments that laws make.

Yeah, and we have systems like this already that make many decisions for ourselves in society that need to be designed in a way that they, like rules about things that sometimes have bad side effects.

And we have to be flexible enough about those rules so that they can be broken when it's obvious that they shouldn't be applied.

So you don't see this on the camera here, but all the decoration in this room is all pictures from 2001, a space odyssey.

So if you were to build HAL 10,000, so an improvement of HAL 9,000, what would you improve?

Well, first of all, I wouldn't ask it to hold secrets and tell lies because that's really what breaks it in the end.

That's the fact that it's asking itself questions about the purpose of the mission.

And it's, you know, pieces things together that it's heard, you know, all the secrecy of the preparation of the mission and the fact that it was a discovery on the lunar surface that really was kept secret.

And 1 part of Hal's memory knows this, and the other part does not know it, and is supposed to not tell anyone, and that creates internal conflict.

So you think there never should be a set of things that an AI system should not be allowed, like a set of facts that should not be shared with the human operators?

Well, I think, no, I think it should be a bit like in the design of autonomous AI systems, there should be the equivalent of the oath that doctors sign up to.

So there's certain things, certain rules that you have to abide by.

And we can sort of hardwire this into our machines to kind of make sure they don't go.

So I'm not, you know, an advocate of the 3 laws of robotics, you know, the Asimov kind of thing, because I don't think it's practical, but, you know, some level of limits.

But to be clear, these are not questions that are kind of really worth asking today, because we just don't have the technology to do this.

We don't have autonomous intelligent machines.

Some are intelligent machines that are very specialized, but they don't really sort of satisfy an objective.

They're just kind of trained to do 1 thing.

So until we have some idea for design of a full-fledged autonomous intelligent system, asking the question of how we design is subjective, I think is a little too abstract.

There's useful elements to it in that it helps us understand our own ethical codes, humans.

So even just as a thought experiment, if you imagine that an AGI system is here today, how would we program it as a kind of nice thought experiment of constructing how should we have a law, have a system of laws for us humans.

And I think there's echoes of that idea too in the AI systems we have today that don't have to be that intelligent, like autonomous vehicles.

These things start creeping in that we're thinking about, but certainly they shouldn't be framed as how.

Looking back, what is the most, I'm sorry if it's a silly question, but what is the most beautiful or surprising idea in deep learning or AI in general that you've ever come across?

Sort of personally, what you said back and just had this kind of, oh, that's pretty cool moment.

I don't know if it's an idea rather than a sort of empirical fact.

The fact that You can build gigantic neural nets, train them on relatively small amounts of data, relatively, with stochastic gradient descent, and that actually works.

Breaks everything you read in every textbook, right?

Every pre-deep learning textbook that told you you need to have fewer parameters than you have data samples.

You know, if you have a non-convex objective function, you have no guarantee of convergence.

You know, all those things that you read in textbook and they tell you to stay away from this and they're all wrong.

And- a huge number of parameters, non-convex, and somehow which is very relative to the number of parameters, data, it's able to learn anything.

Well, it was kind of obvious to me before I knew anything that this is a good idea.

And then it became surprising that it worked because I started reading those textbooks.

So, can you talk through the intuition of why it was obvious to you if you remember?

So, the intuition was, it's sort of like, you know, those people in the late 19th century who proved that heavier than air flight was impossible, right?

And so, on the face of it, it's obviously wrong as an empirical question, right?

And so we have the same kind of thing that, you know, we know that the brain works, we don't know how, but we know it works.

And we know it's a large network of neurons in interaction and that learning takes place by changing the connections.

So kind of getting this level of inspiration without copying the details, but sort of trying to derive basic principles.

You know, that kind of gives you a clue as to which direction to go.

There's also the idea somehow that I've been convinced of since I was an undergrad that, even before, that intelligence is inseparable from learning.

So the idea somehow that you can create an intelligent machine by basically programming, for me, was a non-starter from the start.

Every intelligent entity that we know about arrives at this intelligence through learning.

So learning, machine learning was a completely obvious path.

Also because I'm lazy, so, you know, kind of.

You automate basically everything and learning is the automation of intelligence.

So do you think, so what is learning then?

Because Do you think of reasoning as learning?

Well, reasoning is certainly a consequence of learning as well, just like other functions of the brain.

The big question about reasoning is, how do you make reasoning compatible with gradient-based learning?

Do you think neural networks can be made to reason?

So the question is how much prior structure do you have to put in the neural net so that something like human reasoning will emerge from it, from learning.

Another question is, all of our kind of model of what reasoning is that are based on logic are discrete and are therefore incompatible with gradient-based learning.

And I'm a very strong believer in this idea of gradient-based learning.

I don't believe that other types of learning that don't use kind of gradient information, if you want.

Well, it's not that I don't like it, it's just that it's incompatible with learning and I'm a big fan of learning.

So in fact, that's perhaps 1 reason why deep learning has been kind of looked at with suspicion by a lot of computer scientists, because the math is very different.

The math that you use for deep learning has more to do with cybernetics, the kind of math you do in electrical engineering, than the kind of math you do in computer science.

And nothing in machine learning is exact, right?

Computer science is all about sort of, you know, obsessive compulsive attention to details of like, you know, every index has to be right and you can prove that an algorithm is correct, right?

Machine learning is the science of sloppiness, really.

So, okay, maybe let's feel around in the dark of what is a neural network that reasons or a system that works with continuous functions that's able to do, build knowledge, however we think about reasoning, build on previous knowledge, build on extra knowledge, create new knowledge, generalize outside of any training set ever built, what does that look like?

If, yeah, maybe do you have inklings of thoughts of what that might look like?

If I had precise ideas about this, I think, you know, we'd be building it right now.

But, and there are people working on this whose main research interest is actually exactly that, right?

So what you need to have is a working memory.

So you need to have some device, if you want, some subsystem that can store a relatively large number of factual episodic information for a reasonable amount of time.

So in the brain, for example, there are kind of 3 main types of memory.

1 is the sort of memory of the state of your cortex.

And that sort of disappears within 20 seconds.

You can't remember things for more than about 20 seconds or a minute if you don't have any other form of memory.

The second type of memory, which is longer term, but still short term is the hippocampus.

So you can, you know, you came into this building, you remember where the exit is, where the elevators are.

You have some map of that building that's stored in your hippocampus.

You might remember something about what I said, you know, a few minutes ago.

And then the longer term memory is in the synapse, the synapses, right?

So what you need if you want a system that's capable of reasoning is that you want the hippocampus-like thing, right?

And that's what people have tried to do with memory networks and, you know, neural training machines and stuff like that, right?

And now with transformers, which have sort of a memory in their kind of self-attention system, you can think of it this way.

Another thing you need is some sort of network that can access this memory, get an information back, and then crunch on it, and then do this iteratively multiple times.

Because a chain of reasoning is a process by which you update your knowledge about the state of the world, about what's gonna happen, et cetera.

And that has to be this sort of recurrent operation basically.

And you think that kind of, if we think about a transformer, so that seems to be too small to contain the knowledge that's, to represent the knowledge that's contained in Wikipedia, for example.

But a transformer doesn't have this idea of recurrence.

It's got a fixed number of layers, and that's the number of steps that, you know, limits, basically, its representation.

But recurrence would build on the knowledge somehow.

I mean, it would evolve the knowledge and expand the amount of information perhaps or useful information within that knowledge.

But is this something that just can emerge with size?

Because it seems like everything we have now

I mean, how you access and write into an SOHT memory in an efficient way.

I mean, sort of the original memory network maybe had something like the right architecture, but if you try to scale up a memory network so that the memory contains all Wikipedia, it doesn't quite work.

So there's another form of reasoning, which is true, which is very classical also in some types of AI.

And it's based on, let's call it energy minimization.

Okay, so you have some sort of objective, some energy function that represents the quality or the negative quality.

Energy goes up when things get bad and they get low when things get good.

So let's say you want to figure out what gestures do I need to do to grab an object or walk out the door?

If you have a good model of your own body, a good model of the environment, using this kind of energy minimization, you can do planning.

And in optimal control, it's called model predictive control.

You have a model of what's going to happen in the world as a consequence of your actions.

And that allows you to, by energy minimization, figure out a sequence of action that optimizes a particular objective function, which measures, you know, minimizes the number of times you're going to hit something and the energy you're going to spend doing the gesture and et cetera.

And perhaps what led to the ability of humans to reason is the fact that, or, you know, species, you know, that appear before us had to do some sort of planning to be able to hunt and survive and survive the winter in particular.

And So, you know, it's the same capacity that you need to have.

So in your intuition is, if we look at expert systems and encoding knowledge as logic systems, as graphs, in this kind of way, is not a useful way to think about knowledge?

Graphs are a little brittle or logic representation.

So basically, you know, variables that have values and then constraint between them that are represented by rules is a little too rigid and too brittle.

So some of the early efforts in that respect were to put probabilities on them.

So a rule, if you have this and that symptom, you have this disease with that probability and you should prescribe that antibiotic with that probability, right?

And that's what that branch of AI led to, you know, Bayesian networks and graphical models and causal inference and variational, you know, method.

So there is, I mean, certainly a lot of interesting work going on in this area.

The main issue with this is knowledge acquisition.

How do you reduce a bunch of data to a graph of this type.

Yeah, it relies on the expert to, on the human being to encode, to add knowledge.

The second question is, do you want to represent knowledge as symbols, and do you want to manipulate them with logic?

And again, that's incompatible with learning.

So 1 suggestion, which Geoff Hinton has been advocating for many decades, is replace symbols by vectors.

Think of it as pattern of activities in a bunch of neurons or units or whatever you want to call them.

And replace logic by continuous functions.

There's a very good set of ideas by written in a paper about 10 years ago by Leon Boutou on who is here at Facebook.

The title of the paper is from machine learning to machine reasoning.

And his idea is that a learning system should be able to manipulate objects that are in a space and then put the result back in the same space.

So it's this idea of working memory basically.

And in a sense, that might learn something like the simple expert systems.

I mean, you can learn basic logic operations there.

There's a big debate on sort of how much prior structure you have to put in for this kind of stuff to emerge.

That's the debate I have with Gary Marcus and people like that.

So, and the other person, so I just talked to Judea Pearl, from the mentioned causal inference world.

So his worry is that the current neural networks are not able to learn what causes what causal inference between things.

So I think, I think he's right and wrong about this.

If he's talking about the sort of classic type of neural nets, people sort of didn't worry too much about this.

But there's a lot of people now working on causal inference, and there's a paper that just came out last week by Leon Boutou, among others, David Lopez-Paz and a bunch of other people, Exactly on that problem of how do you kind of, you know, get a neural net to sort of pay attention to real causal relationships, which may also solve issues of bias in data and things like this.

I'd like to read that paper because that ultimately, the challenge there is also seems to fall back on the human expert to ultimately decide causality between things.

People are not very good at establishing causality, first of all.

So first of all, you talk to physicists, and physicists actually don't believe in causality because look at all the basic laws of macrophysics are time reversible, so there's no causality.

It's as soon as you start looking at macroscopic systems where there is unpredictable randomness, where there is clearly an arrow of time, but it's a big mystery in physics actually, how that emerges.

Is it emergent or is it part of the fundamental fabric of reality?

Or is it a bias of intelligent systems that because of the second law of thermodynamics, we perceive a particular hour of time, but in fact, it's kind of arbitrary, right?

So yeah, physicists, mathematicians, they don't care about, I mean, the math doesn't care about the flow of time.

Well, certainly, certainly microphysics doesn't.

People themselves are not very good at establishing causal relationships.

If you ask, I think it was in 1 of Seymour Papert's book on like children learning, you know, he studied with Jean Piaget, you know, he's the guy who co-authored the book Perceptron with Marvin Minsky that kind of killed the first wave of neural nets.

He in the sense of studying learning in humans and machines, that's why he got interested in perceptron.

And he wrote that if you ask a little kid about what is the cause of the wind?

A lot of kids will say, they will think for a while and they'll say, oh, it's the branches in the trees, they move and that creates wind, right?

So they get the causal relationship backwards.

And it's because their understanding of the world and intuitive physics is not that great, right?

I mean, these are like 4 or 5 year old kids.

You know, it gets better and then you understand that this, it can be, right?

But there are many things which we can, because of our common sense understanding of things, what people call common sense, and our understanding of physics, we can, there's a lot of stuff that we can figure out causality, even with diseases, we can figure out what's not causing what, often.

There's a lot of mystery, of course, but the idea is that you should be able to encode that into systems, because it seems unlikely they'd be able to figure that out themselves.

Well, whenever we can do intervention, but all of humanity has been completely deluded for millennia, probably since its existence, about a very, very wrong causal relationship where whatever you can explain, you attribute it to some deity, some divinity, right?

That's a way of saying, like, I don't know the cause, so, you know, God did it, right?

So you mentioned Marvin Minsky and the irony of, you know, maybe causing the first AI winter.

You were there in the 90s, you were there in the 80s, of course, in the 90s, why do you think people lost faith in deep learning in the 90s and founded again a decade later, over a decade later.

Yeah, it wasn't called deep learning yet.

I mean, I think I would put that around 1995, at least the machine learning community.

There was always a neural net community, but it became kind of disconnected from sort of mainstream machine learning, if you want.

There were, it was basically electrical engineering that kept at it.

And computer science gave up on neural nets.

I was too close to it to really sort of analyze it with sort of an unbiased eye, if you want.

So the first 1 is at the time neural nets were, it was very hard to make them work in the sense that you would, you know, implement backprop in your favorite language.

And that favorite language was not Python.

It was not any of those things because they didn't exist.

You had to write it in Fortran or C or something like this.

You would probably make some very basic mistakes, like, you know, barely initialize your weights, make the network too small because you read in the textbook, you know, you don't want too many parameters, right?

And of course, you know, and you would train on XOR because you didn't have any other data set to try it on.

And of course, you know, it works half the time.

Also, you're trying it with batch gradient, which, you know, isn't really sufficient.

So there's a lot of, there's a bag of tricks that you had to know to make those things work or you had to reinvent.

And a lot of people just didn't and they just couldn't make it work.

The investment in software platform to be able to kind of display things, figure out why things don't work, kind of get a good intuition for how to get them to work, have enough flexibility so you can create, you know, network architectures like convolutional nets and stuff like that.

I mean, you had to write everything from scratch.

And again, you didn't have any Python or Matlab or anything, right?

See all Lex Fridman transcripts on Youtube

Yann LeCun: Deep Learning, ConvNets, and Self-Supervised Learning | Lex Fridman Podcast #36