This is a clip from a conversation with Gary Marcus from Oct 2019. New full episodes every Mon & Thu and 1-2 new clips or a new non-podcast video on all other days. You can watch the full conversation here: https://www.youtube.com/watch?v=vNOTDn3D_RI
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on iTunes:
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a few clips every Tue & Fri. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Gary Marcus is a professor emeritus at NYU, founder of Robust.AI and Geometric Intelligence, the latter is a machine learning company acquired by Uber in 2016. He is the author of several books on natural and artificial intelligence, including his new book Rebooting AI: Building Machines We Can Trust. Gary has been a critical voice highlighting the limits of deep learning and discussing the challenges before the AI community that must be solved in order to achieve artificial general intelligence.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

You've talked about this, you've written about it, you've thought about it, nature versus nurture.

So what innate knowledge do you think we're born with and what do we learn along the way in those early months and years?

Can I just say how much I like that question?

You phrased it just right and almost nobody ever does.

Which is what is the innate knowledge and what's learned along the way?

So many people dichotomize it and they think it's nature versus nurture.

When it is obviously has to be nature and nurture.

You can't learn the stuff along the way unless you have some innate stuff.

But just because you have the innate stuff doesn't mean you don't learn anything.

And so many people get that wrong, including in the field.

People think if I work in machine learning, the learning side, I must not be allowed to work on the innate side where that will be cheating.

I've talked to folks who studied the development of the brain, and the growth of the brain in the first few days, in the first few months, in the womb, all of that, Is that innate?

So that process of development from a stem cell to the growth of the central nervous system and so on, to the information that's encoded through the long arc of evolution.

So all of that comes into play and it's unclear.

It's not just whether it's a dichotomy or not, it's where most or where the knowledge is encoded.

So what's your intuition about the innate knowledge, the power of it, what's contained in it, what can we learn from it?

1 of my earlier books was actually trying to understand the biology of this.

The book was called The Birth of the Mind.

Like how is it the genes even build innate knowledge?

And from the perspective of the conversation we're having today, there's actually 2 questions.

1 is what innate knowledge or mechanisms or what have you people or other animals might be endowed with.

I always like showing this video of a baby Ibex climbing down a mountain.

That baby Ibex, a few hours after his birth, knows how to climb down a mountain, that means that it knows not consciously something about its own body and physics and 3D geometry and all of this kind of stuff.

So there's 1 question about like, what does biology give its creatures?

The question I thought about in the book, The Birth of the Mind.

And then there's a question of what AI should have.

But I would say that, you know, it's a pretty interesting set of things that we are equipped with that allows us to do a lot of interesting things.

So I would argue or guess based on my reading of the developmental psychology literature, which I've also participated in, that children are born with a notion of space, time, other agents, places, and also this kind of mental algebra that I was describing before.

No certain causation if I didn't just say that.

They're like frameworks for learning the other things.

Are they disjoint in your view or is it just somehow all connected?

Is it all kind of connected in some mesh that's language-like of understanding concepts altogether?

I don't think we know for people how they're represented and machines just don't really do this yet.

So I think it's an interesting open question both for science and for engineering.

Some of it has to be at least interrelated in the way that like the interfaces of a software package have to be able to talk to 1 another.

So the systems that represent space and time can't be totally disjoint because a lot of the things that we reason about are the relations between space and time and cause.

So I put this on and I have expectations about what's gonna happen with the bottle cap on top of the bottle.

You know, if the cap is over here, I get a different outcome.

If the timing is different, if I put this here, after I move that, then I get a different outcome that relates to causality.

So obviously these mechanisms, whatever they are, can certainly communicate with each other.

So I think evolution had a significant role to play in the development of this whole colluge, right?

Oh, it's terribly inefficient, except that.

It's inefficient except that once it gets a good idea, it runs with it.

So it took, I guess, a billion years, if I went, roughly a billion years to evolve to a vertebrate brain plan.

And once that vertebrate brain plan evolved, it spread everywhere.

So fish have it and dogs have it and we have it.

We have adaptations of it and specializations of it.

But, and the same thing with a primate brain plan.

So monkeys have it and apes have it and we have it.

So, you know, there are additional innovations like color vision and those spread really rapidly.

So, it takes evolution a long time to get a good idea, but, you know, being anthropomorphic and not literal here, But once it has that idea, so to speak, which caches out into 1 set of genes or in the genome, those genes spread very rapidly.

And they're like subroutines or libraries, I guess is the word people might use nowadays or be more familiar with.

They're libraries that can get used over and over again.

So once you have the library for building something with multiple digits, you can use it for a hand, but you can also use it for a foot.

You just kind of reuse the library with slightly different parameters.

Evolution does a lot of that, which means that the speed over time picks up.

So evolution can happen faster because you have bigger and bigger libraries.

And what I think has happened in attempts at evolutionary computation is that people start with libraries that are very, very minimal, like almost nothing.

And then progress is slow and it's hard for someone to get a good PhD thesis out of it and then give up.

If we had richer libraries to begin with, if you were evolving from systems that had an originate structure to begin with, then things might speed up.

Or more PhD students, if the evolutionary process is indeed in a meta way, runs away with good ideas, you need to have a lot of ideas, pool of ideas, in order for it to discover 1 that you can run away with.

And PhD students representing individual ideas as well.

Yeah, I mean, you could throw a billion PhD students at it.

Yeah, the monkeys are typewriters with Shakespeare, yeah.

Well, I mean, those aren't cumulative, right?

And part of the point that I'm making is that evolution is cumulative.

So if you have a billion monkeys independently, you don't really get anywhere.

But if you have a billion monkeys, and I think Dawkins made this point originally, or probably other people, but Dawkins made it very nice, and either a selfish teen or blind watchmaker.

If there's some sort of fitness function that can drive you towards something, I guess that's Dawkins' point, and my point, which is a variation on that, is that if the evolution is cumulative, I mean, they're related points, then you can start going faster.

Do you think something like the process of evolution is required to build intelligent systems?

So all the stuff that evolution did, a good engineer might be able to do.

So for example, evolution made quadrupeds, which distribute the load across a horizontal surface.

A good engineer could come up with that idea.

I mean, sometimes good engineers come up with ideas by looking at biology.

Part of what I'm suggesting is we should look at biology a lot more.

We should look at the biology of thought and understanding the biology by which creatures intuitively reason about physics or other agents or like how do dogs reason about people?

If we could understand, at my college we joked dognition, if we could understand dognition well and how it was implemented, that might help us with our AI.

So do you think it's possible that the kind of time scale that evolution took is the kind of time scale that will be needed to build intelligent systems, or can we significantly accelerate that process inside a computer?

mean, I think the way that we accelerate that process is we borrow from biology.

Not slavishly, but I think we look at how biology has solved problems and we say, does that inspire any engineering solutions here?

Try to mimic biological systems and then therefore have a shortcut.

Yeah, I mean, there's a field called biomimicry, and people do that for material science all the time.

We should be doing the analog of that for AI, and the analog for that for AI is to look at cognitive science, or the cognitive sciences, which is psychology, maybe neuroscience, linguistics, and so forth, look to those for insight.


Speaker 0: You've talked about this, you've written about it, you've thought about it, nature versus nurture. So what innate knowledge do you think we're born with and what do we learn along the way in those early months and years?

Speaker 1: Can I just say how much I like that question? You phrased it just right and almost nobody ever does. Which is what is the innate knowledge and what's learned along the way? So many people dichotomize it and they think it's nature versus nurture. When it is obviously has to be nature and nurture.

They have to work together. You can't learn the stuff along the way unless you have some innate stuff. But just because you have the innate stuff doesn't mean you don't learn anything. And so many people get that wrong, including in the field. People think if I work in machine learning, the learning side, I must not be allowed to work on the innate side where that will be cheating.

Exactly, people have said that to me. And it's just absurd, so thank you.

Speaker 0: But you could break that apart more. I've talked to folks who studied the development of the brain, and the growth of the brain in the first few days, in the first few months, in the womb, all of that, Is that innate? So that process of development from a stem cell to the growth of the central nervous system and so on, to the information that's encoded through the long arc of evolution. So all of that comes into play and it's unclear. It's not just whether it's a dichotomy or not, it's where most or where the knowledge is encoded.

So what's your intuition about the innate knowledge, the power of it, what's contained in it, what can we learn from it?

Speaker 1: 1 of my earlier books was actually trying to understand the biology of this. The book was called The Birth of the Mind. Like how is it the genes even build innate knowledge? And from the perspective of the conversation we're having today, there's actually 2 questions. 1 is what innate knowledge or mechanisms or what have you people or other animals might be endowed with.

I always like showing this video of a baby Ibex climbing down a mountain. That baby Ibex, a few hours after his birth, knows how to climb down a mountain, that means that it knows not consciously something about its own body and physics and 3D geometry and all of this kind of stuff. So there's 1 question about like, what does biology give its creatures? And what is evolved in our brains? How is that represented in our brains?

The question I thought about in the book, The Birth of the Mind. And then there's a question of what AI should have. And they don't have to be the same. But I would say that, you know, it's a pretty interesting set of things that we are equipped with that allows us to do a lot of interesting things. So I would argue or guess based on my reading of the developmental psychology literature, which I've also participated in, that children are born with a notion of space, time, other agents, places, and also this kind of mental algebra that I was describing before.

No certain causation if I didn't just say that. So at least those kinds of things. They're like frameworks for learning the other things.

Speaker 0: Are they disjoint in your view or is it just somehow all connected? You've talked a lot about language. Is it all kind of connected in some mesh that's language-like of understanding concepts altogether?

Speaker 1: I don't think we know for people how they're represented and machines just don't really do this yet. So I think it's an interesting open question both for science and for engineering. Some of it has to be at least interrelated in the way that like the interfaces of a software package have to be able to talk to 1 another. So the systems that represent space and time can't be totally disjoint because a lot of the things that we reason about are the relations between space and time and cause. So I put this on and I have expectations about what's gonna happen with the bottle cap on top of the bottle.

And those span space and time. You know, if the cap is over here, I get a different outcome. If the timing is different, if I put this here, after I move that, then I get a different outcome that relates to causality. So obviously these mechanisms, whatever they are, can certainly communicate with each other.

Speaker 0: So I think evolution had a significant role to play in the development of this whole colluge, right? How efficient do you think is evolution?

Speaker 1: Oh, it's terribly inefficient, except that. Well, can we do better? Well, I'll come to that in a second. It's inefficient except that once it gets a good idea, it runs with it. So it took, I guess, a billion years, if I went, roughly a billion years to evolve to a vertebrate brain plan.

And once that vertebrate brain plan evolved, it spread everywhere. So fish have it and dogs have it and we have it. We have adaptations of it and specializations of it. But, and the same thing with a primate brain plan. So monkeys have it and apes have it and we have it.

So, you know, there are additional innovations like color vision and those spread really rapidly. So, it takes evolution a long time to get a good idea, but, you know, being anthropomorphic and not literal here, But once it has that idea, so to speak, which caches out into 1 set of genes or in the genome, those genes spread very rapidly. And they're like subroutines or libraries, I guess is the word people might use nowadays or be more familiar with. They're libraries that can get used over and over again. So once you have the library for building something with multiple digits, you can use it for a hand, but you can also use it for a foot.

You just kind of reuse the library with slightly different parameters. Evolution does a lot of that, which means that the speed over time picks up. So evolution can happen faster because you have bigger and bigger libraries. And what I think has happened in attempts at evolutionary computation is that people start with libraries that are very, very minimal, like almost nothing. And then progress is slow and it's hard for someone to get a good PhD thesis out of it and then give up.

If we had richer libraries to begin with, if you were evolving from systems that had an originate structure to begin with, then things might speed up.

Speaker 0: Or more PhD students, if the evolutionary process is indeed in a meta way, runs away with good ideas, you need to have a lot of ideas, pool of ideas, in order for it to discover 1 that you can run away with. And PhD students representing individual ideas as well.

Speaker 1: Yeah, I mean, you could throw a billion PhD students at it.

Speaker 0: Yeah, the monkeys are typewriters with Shakespeare, yeah.

Speaker 1: Well, I mean, those aren't cumulative, right? That's just random. And part of the point that I'm making is that evolution is cumulative. So if you have a billion monkeys independently, you don't really get anywhere. But if you have a billion monkeys, and I think Dawkins made this point originally, or probably other people, but Dawkins made it very nice, and either a selfish teen or blind watchmaker.

If there's some sort of fitness function that can drive you towards something, I guess that's Dawkins' point, and my point, which is a variation on that, is that if the evolution is cumulative, I mean, they're related points, then you can start going faster.

Speaker 0: Do you think something like the process of evolution is required to build intelligent systems? So if we-

Speaker 1: Not logically. So all the stuff that evolution did, a good engineer might be able to do. So for example, evolution made quadrupeds, which distribute the load across a horizontal surface. A good engineer could come up with that idea. I mean, sometimes good engineers come up with ideas by looking at biology.

There's lots of ways to get your ideas. Part of what I'm suggesting is we should look at biology a lot more. We should look at the biology of thought and understanding the biology by which creatures intuitively reason about physics or other agents or like how do dogs reason about people? Like they're actually pretty good at it. If we could understand, at my college we joked dognition, if we could understand dognition well and how it was implemented, that might help us with our AI.

Speaker 0: So do you think it's possible that the kind of time scale that evolution took is the kind of time scale that will be needed to build intelligent systems, or can we significantly accelerate that process inside a computer? I

Speaker 1: mean, I think the way that we accelerate that process is we borrow from biology. Not slavishly, but I think we look at how biology has solved problems and we say, does that inspire any engineering solutions here?

Speaker 0: Try to mimic biological systems and then therefore have a shortcut.

Speaker 1: Yeah, I mean, there's a field called biomimicry, and people do that for material science all the time. We should be doing the analog of that for AI, and the analog for that for AI is to look at cognitive science, or the cognitive sciences, which is psychology, maybe neuroscience, linguistics, and so forth, look to those for insight.

See all Lex Fridman transcripts on Youtube

Gary Marcus: Nature vs Nurture is a False Dichotomy | AI Podcast Clips