Full episode with John Hopfield (Feb 2020): https://www.youtube.com/watch?v=DKyzcbNr8WE
Clips channel (Lex Clips): https://www.youtube.com/lexclips
Main channel (Lex Fridman): https://www.youtube.com/lexfridman
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on Apple Podcasts (iTunes):
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

John Hopfield is professor at Princeton, whose life's work weaved beautifully through biology, chemistry, neuroscience, and physics. Most crucially, he saw the messy world of biology through the piercing eyes of a physicist. He is perhaps best known for his work on associate neural networks, now known as Hopfield networks that were one of the early ideas that catalyzed the development of the modern field of deep learning.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

You can't just put that on another shelf.

Every once in a while I get interested in consciousness and then I go and I've done that for years and ask 1 of my betters, as it were, their view on consciousness.

Let's try to take a brief step into that room.

Well, I asked Marvin Minsky, did you go into consciousness?

And Marvin said, consciousness is basically overrated.

After all, all the things your brain does, which are actually hard computations, you do non-consciously.

And there's so much evidence that even the simple things you do, you can make committed decisions about them.

The neurobiologist can say, He's now committed, he's going to move the hand left before you know it.

So his view that consciousness is not, that's just like little icing on the cake.

Non-conscious, what's the better word, sir?

It's only that Freud captured the other word.

Yeah, it's a confusing word, subconscious.

Nicholas Chater wrote an interesting book, I think the title of it is, The Mind is Flat.

Flat, in a neural net sense, might be flat is something which is a very broad neural

net without really any layers in depth, or the deep brain would be many layers and not so broad.

In the same sense that if you push Minsky hard enough, he would probably have said, consciousness is your effort to explain to yourself that

Yeah, it's the weaving of the narrative around the things that

And then so much of what we do for our memories of events, for example, if there's some traumatic event you witness, you will have a few facts about it correctly done.

If somebody asks you about it, you will weave a narrative, which is actually much more rich in detail than that, based on some anchor points you have of correct things

and pulling together general knowledge on the other, but you will have a narrative.

And once you generate that narrative, you are very likely to repeat that narrative and claim that all the things you have in it are actually the correct things.

the Watergate slash impeachment era of John Dean.

John Dean, you're too young to know, had been the personal lawyer of Nixon.

And so John Dean was involved in the cover-up, and John Dean ultimately realized the only way to keep himself out of jail for a long time was actually to tell some of the truths about Nixon.

in great detail, and very convincing detail.

And long afterward, Some of the tapes, the secret tapes as it were, from which these, Don was, Gene was recalling these conversations were published.

And 1 found out that John Dean had a good, but not exceptional memory.

What he had was an ability to paint vividly and in some sense accurately the tone of what was going on.

By the way, that's a beautiful description of consciousness.

Do you, like where do you stand in your, today?

So perhaps it changes day to day, but where do you stand on the importance of consciousness in our whole big mess of cognition.

Is it just a little narrative maker, or is it actually fundamental to intelligence?

When I asked Francis Crick about consciousness, he launched forward in a long monologue about Mendel and the peas.

And how Mendel knew that there was something, and how biologists understood that there was

something in inheritance, which was just very, very different, and the fact that inherited traits didn't just wash out into a gray, but were this or this, and propagated, that that was absolutely fundamental to biology.

And it took generations of biologists to understand that there was genetics, and it took another generation or 2 to understand that genetics came from DNA.

But very shortly after Mendel, thinking biologists did realize that there was a deep problem about inheritance.

And Francis would have liked to have said, and that's why I'm working on consciousness.

But of course, he didn't have any smoking gun in the sense of Mendel.

If you read his book, which he wrote with Koch, I think.

I find it unconvincing for the smoking gun reason.

So I've gone on collecting views without actually having taken a very strong 1 myself, because I haven't seen the entry point.

the point of view of physics, I don't see the entry point.

Whereas in neurobiology, once I understood the idea of a collective, an evolution of dynamics, which could be described as a collective phenomenon, I thought, ah, there's a point where what I know about physics is so different from any neurobiologist that I have something I might be able to contribute.

And right now there's no way to grasp at consciousness from a physics perspective.

And of course people, physicists like everybody else, think very muddily about things.

You ask the closely related question about free will.

Physicists will give an offhand answer and then backtrack, backtrack, backtrack where

they realize that the answer they gave must fundamentally contradict the laws of physics.

Answering questions of free will and consciousness naturally lead to contradictions from a physics perspective.

Because it eventually ends up with quantum mechanics and then you get into that whole mess of trying to understand how much, from a physics perspective, how much is determined, already predetermined, much is already deterministic about our universe.

And if you don't push quite that far, you can say, essentially all of neural biology, which is relevant, can be captured by classical equations of motion.

Because in my view of the mysteries of the brain are not the mysteries of quantum mechanics, but the mysteries of what can happen when you have a dynamical system, driven system, with 10 to the 14 parts.

That that complexity is something which is,

that the physics of complex systems is at least as badly understood as the physics of phase coherence in quantum mechanics.


Speaker 0: Let alone consciousness.

Speaker 1: Oh, let alone consciousness, yes, yes.

Speaker 0: Because that's tied up in there too. You can't just put that on another shelf.

Speaker 1: Every once in a while I get interested in consciousness and then I go and I've done that for years and ask 1 of my betters, as it were, their view on consciousness. It's been interesting collecting them.

Speaker 0: What is consciousness? Let's try to take a brief step into that room.

Speaker 2: Well, I asked Marvin Minsky, did you go into consciousness? And Marvin said, consciousness is basically overrated. It may be an epiphenomenon. After all, all the things your brain does, which are actually hard computations, you do non-consciously.

Speaker 1: And there's so much evidence that even the simple things you do, you can make committed decisions about them. The neurobiologist can say, He's now committed, he's going to move the hand left before you know it.

Speaker 3: So his view that consciousness is not, that's just like little icing on the cake. The real cake is in the subconscious.

Speaker 2: Yeah, yeah. Subconscious, non-conscious.

Speaker 1: Non-conscious, what's the better word, sir?

Speaker 2: It's only that Freud captured the other word.

Speaker 3: Yeah, it's a confusing word, subconscious.

Speaker 1: Nicholas Chater wrote an interesting book, I think the title of it is, The Mind is Flat.

Speaker 2: Flat, in a neural net sense, might be flat is something which is a very broad neural

Speaker 1: net without really any layers in depth, or the deep brain would be many layers and not so broad.

Speaker 2: In the same sense that if you push Minsky hard enough, he would probably have said, consciousness is your effort to explain to yourself that

Speaker 1: which you have already done.

Speaker 0: Yeah, it's the weaving of the narrative around the things that

Speaker 3: have already been computed for you.

Speaker 2: That's right. And then so much of what we do for our memories of events, for example, if there's some traumatic event you witness, you will have a few facts about it correctly done. If somebody asks you about it, you will weave a narrative, which is actually much more rich in detail than that, based on some anchor points you have of correct things

Speaker 1: and pulling together general knowledge on the other, but you will have a narrative. And once you generate that narrative, you are very likely to repeat that narrative and claim that all the things you have in it are actually the correct things. There was a marvelous example of that in

Speaker 2: the Watergate slash impeachment era of John Dean. John Dean, you're too young to know, had been the personal lawyer of Nixon.

Speaker 1: And so John Dean was involved in the cover-up, and John Dean ultimately realized the only way to keep himself out of jail for a long time was actually to tell some of the truths about Nixon. And John Dean was a tremendous witness. He would remember these conversations

Speaker 2: in great detail, and very convincing detail. And long afterward, Some of the tapes, the secret tapes as it were, from which these, Don was, Gene was recalling these conversations were published.

Speaker 1: And 1 found out that John Dean had a good, but not exceptional memory. What he had was an ability to paint vividly and in some sense accurately the tone of what was going on.

Speaker 0: By the way, that's a beautiful description of consciousness. Do you, like where do you stand in your, today? So perhaps it changes day to day, but where do you stand on the importance of consciousness in our whole big mess of cognition. Is it just a little narrative maker, or is it actually fundamental to intelligence?

Speaker 1: That's a very hard 1. When I asked Francis Crick about consciousness, he launched forward in a long monologue about Mendel and the peas. And how Mendel knew that there was something, and how biologists understood that there was

Speaker 2: something in inheritance, which was just very, very different, and the fact that inherited traits didn't just wash out into a gray, but were this or this, and propagated, that that was absolutely fundamental to biology. And it took generations of biologists to understand that there was genetics, and it took another generation or 2 to understand that genetics came from DNA.

Speaker 1: But very shortly after Mendel, thinking biologists did realize that there was a deep problem about inheritance. And Francis would have liked to have said, and that's why I'm working on consciousness. But of course, he didn't have any smoking gun in the sense of Mendel. And that's the weakness of his position. If you read his book, which he wrote with Koch, I think.

Speaker 0: Yeah, Christoph Koch, yeah.

Speaker 1: I find it unconvincing for the smoking gun reason.

Speaker 2: So I've gone on collecting views without actually having taken a very strong 1 myself, because I haven't seen the entry point. Not seeing the smoking gun, from

Speaker 1: the point of view of physics, I don't see the entry point. Whereas in neurobiology, once I understood the idea of a collective, an evolution of dynamics, which could be described as a collective phenomenon, I thought, ah, there's a point where what I know about physics is so different from any neurobiologist that I have something I might be able to contribute.

Speaker 0: And right now there's no way to grasp at consciousness from a physics perspective.

Speaker 2: From my point of view, that's correct. And of course people, physicists like everybody else, think very muddily about things. You ask the closely related question about free will. Do you believe you have free will? Physicists will give an offhand answer and then backtrack, backtrack, backtrack where

Speaker 1: they realize that the answer they gave must fundamentally contradict the laws of physics.

Speaker 0: Answering questions of free will and consciousness naturally lead to contradictions from a physics perspective.

Speaker 3: Because it eventually ends up with quantum mechanics and then you get into that whole mess of trying to understand how much, from a physics perspective, how much is determined, already predetermined, much is already deterministic about our universe. There's lots of different-

Speaker 2: And if you don't push quite that far, you can say, essentially all of neural biology, which is relevant, can be captured by classical equations of motion. Right. Because in my view of the mysteries of the brain are not the mysteries of quantum mechanics, but the mysteries of what can happen when you have a dynamical system, driven system, with 10 to the 14 parts.

Speaker 1: That that complexity is something which is,

Speaker 2: that the physics of complex systems is at least as badly understood as the physics of phase coherence in quantum mechanics.

See all Lex Fridman transcripts on Youtube

Consciousness is an Explanation of What Already Has Been Computed (John Hopfield) | AI Podcast Clips