The following is a conversation with Eric Weinstein.

He's a mathematician, economist, physicist, and the managing director of Thiel Capital.

He coined the term, and you could say, is the founder of the intellectual dark web, which is a loosely assembled group of public intellectuals that includes Sam Harris, Jordan Peterson, Steven Pinker, Joe Rogan, Michael Shermer, and a few others.

This conversation is part of the Artificial Intelligence Podcast at MIT and beyond.

If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D.

And now, here's my conversation with Eric Weinstein.

You mentioned Kung Fu Panda as 1 of your favorite movies.

It has the usual profound master-student dynamic going on.

So who has been a teacher that significantly influenced the direction of your thinking and life's work.

So if you're the Kung Fu Panda, who was your Shifu?

Oh, that's interesting because I didn't see Shifu as being the teacher.

And the first conversation sort of doesn't count.

So the magic of the film, in fact, its point is that the teaching that really matters is transferred during a single conversation.

I would say either my grandfather, Harry Rubin and his wife, Sophie Rubin, my grandmother, or Tom Lehrer.

If you give a child Tom Lehrer records, what you do is you destroy their ability to be taken over by later malware.

And it's so irreverent, so witty, so clever, so obscene that it destroys the ability to lead a normal life for many people.

So if I meet somebody who's usually really shifted from any kind of neurotypical presentation, I'll often ask them, are you a Tom Lehrer fan?

And the odds that they will respond are quite high.

Now Tom Lehrer's Poisoning Pigeons in the Park, Tom Lehrer?

There are a small number of Tom Lehrer songs that broke into the general population.

Poisoning Pigeons in the Park, the Element song, and perhaps the Vatican rag.

So when you meet somebody who knows those songs, but doesn't know.

Oh, you're judging me right now, aren't you?

So undoubtedly you know Nikolai Ivanovich Lobachevsky, that song.

So that was a song about plagiarism that was in fact plagiarized, which most people don't know from Danny Kaye, where Danny Kaye did a song called Stanislavski of the Muskie Arts.

And so Tom Lehrer did this brilliant job of plagiarizing a song about, and making it about plagiarism, and then making it about this mathematician who worked in non-Euclidean geometry.

It was extremely addictive, and eventually led me to a lot of different places, 1 of which may have been a PhD in mathematics.

And he was also at least a lecturer in mathematics, I believe, at Harvard, something like that.

When my son turned 13, we didn't tell him, but his bar mitzvah present was dinner with his hero, Tom Lehrer.

And Tom Lehrer was 88 years old, sharp as a tack, irreverent and funny as hell.

And just, you know, there are very few people in this world that you have to meet while they're still here and that was definitely 1 for our family.

reflection of intelligence in some kind of deep way, like where that would be a good test of intelligence whether you're a Tom Lehrer fan.

So what do you think that is about wit, about that kind of humor, ability to see the absurdity in existence, do you think that's connected to intelligence or are we just 2 Jews on a mic that appreciate that kind of humor?

No, I think that it's absolutely connected to intelligence.

You can see it, there's a place where Tom Lehrer decides that he's going to lampoon Gilbert of Gilbert and Sullivan and he's going to outdo Gilbert with clever meaningless wordplay.

He's doing Clementine as if Gilbert and Sullivan wrote it.

And he says, that I mister depressed her young sister named mister, this mister to pester she tried.

Pestering sisters of festering blister, you best to resist her, say I.

The sister persisted, the mister resisted, I kissed her all loyalty slip.

When she said I could have her, her sister's cadaver must surely have turned in its crypt.

That's so dense, it's so insane, that that's clearly intelligence because it's hard to construct something like that.

If I look at my favorite Tom Lyric, Tom Lyric lyric, you know, there's a perfectly absurd 1, which is once all the Germans were warlike and mean, but that couldn't happen again.

We taught them a lesson in 1918 and they've hardly bothered us since then.

That is a different kind of intelligence.

You're taking something that is so horrific and you're sort of making it palatable and funny and demonstrating also just your humanity.

I mean, I think the thing that came through as Tom Lehrer wrote all of these terrible, horrible lines was just what a sensitive and beautiful soul he was who was channeling pain through humor and through grace.

I've seen throughout Europe, throughout Russia, that same kind of humor emerged from the generation of World War II.

It seemed like that humor is required to somehow deal with the pain and the suffering of that that war created.

Well, you do need the environment to create the broad Slavic soul.

I don't think that many Americans really appreciate Russian humor, how you had to joke during the time of, let's say, Article 58 under Stalin.

You know, the concept of a Russian satirical magazine like Krokodil doesn't make sense.

So you have this cross-cultural problem that There are certain areas of human experience that it would be better to know nothing about.

And quite unfortunately, Eastern Europe knows a great deal about them, which makes the songs of Vladimir Vesotsky so potent, the prose of Pushkin, whatever it is, you have to appreciate the depth of the Eastern European experience.

And I would think that perhaps Americans knew something like this around the time of the Civil War or maybe under slavery and Jim Crow, or even the harsh tyranny of the coal and steel employers during the labor wars.

But in general, I would say it's hard for us to understand and imagine the collective culture Unless we have the system of selective pressures that for example Russians were subjected to Yes, if there is 1 good thing that comes out of war its literature art

I think almost everything is good about war except for death and destruction.

Without the death, it would bring the romance of it.

Well, this is why we're always caught up in war.

We have this very ambiguous relationship to it is that it makes life real and pressing and meaningful and at an unacceptable price and the price has never been higher.

In 1 of the conversation you had, or 1 of the videos, you described that 1 of the things AI systems can't do, and biological systems can, is self-replicate in the physical world.

Well, yes, the physical robots can't self-replicate.

This is a very tricky point, which is that the only thing that we've been able to create that's really complex, that has an analog of our reproductive system, is software.

But nevertheless, software replicates itself, if we're speaking strictly for the replication, in this kind of digital space.

So let me, just to begin, let me ask a question.

Do you see a protective barrier or a gap between the physical world and the digital world?

Let's call it the logical world versus the physical world.

Well, because even though we had, let's say Einstein's brain preserved, it was meaningless to us as a physical object because we couldn't do anything with what was stored in it at a logical level.

And so the idea that something may be stored logically and that it may be stored physically are not necessarily, we don't always benefit from synonymizing.

I'm not suggesting that there isn't a material basis to the logical world, but that it does warrant identification with a separate layer that need not invoke logic gates and zeros and ones.

And So connecting those 2 worlds, the logical world and the physical world, or maybe just connecting to the logical world inside our brain, Einstein's brain.

You mentioned the idea of out, outtelligence.

Yes, this is the only essay that John Brockman ever invited me to write that he refused to publish in Edge.

Well, maybe it wasn't well written, but I don't know.

It's quite unique and new, and at least from my view, stance point, maybe you can explain it.

What I was thinking about is why it is that we're waiting to be terrified by artificial general intelligence, when in fact artificial life is terrifying in and of itself and it's already here.

So in order to have a system of selective pressures, you need 3 distinct elements.

You need variation within a population, you need heritability, and you need differential success.

So what's really unique, and I've made this point I think elsewhere, about software is that if you think about what humans know how to build, that's impressive.

So I always take a car and I say, does it have an analog of each of the physical physiological systems?

It has an onboard computer, has a digestive system.

The 1 thing it doesn't have is a reproductive system.

But if you can call spawn on a process, effectively you do have a reproductive system.

And that means that you can create something with variation, heritability, and differential success.

Now, the next step in the chain of thinking was, where do we see inanimate, non-intelligent life outwitting intelligent life?

And I have 2 favorite systems, and I try to stay on them so that we don't get distracted.

1 of which is the ophrys orchid subspecies, or subclade, I don't know what to call it.

Yeah, it's a type of flower that mimics the female of a pollinator species in order to dupe the males into engaging, it was called pseudocopulation with the fake female, which is usually represented by the lowest petal.

And there's also a pheromone component to fool the males into thinking they have a mating opportunity.

But the flower doesn't have to give up energy in the form of nectar as a lure because it's tricking the males.

The other system is a particular species of mussel, Lampicillus in the clear streams of Missouri.

And it fools bass into biting a fleshy lip that contain its young.

And when the bass see this fleshy lip, which looks exactly like a species of fish that the bass like to eat, the young explode and clamp onto the gills and parasitize the bass and also lose the bass to redistribute them as they eventually release.

Both of these systems, you have a highly intelligent dupe being fooled by a lower life form.

And what is sculpting these, these convincing lures?

It's the intelligence of previously duped targets for these strategies.

So when the target is smart enough to avoid the strategy, those weaker mimics fall off.

They have terminal lines and only the better ones survive.

So it's an arms race between the target species that is being parasitized getting smarter and this other less intelligent or non-intelligent object getting as if smarter.

And so what you see is that artificial general intelligence is not needed to parasitize us.

It's simply sufficient for us to outwit ourselves.

So you could have a program, let's say, you know, 1 of these Nigerian scams, that writes letters and uses whoever sends it Bitcoin to figure out which aspects of the program should be kept, which should be varied and thrown away.

And you don't need it to be in any way intelligent in order to have a really nightmarish scenario of being parasitized by something that has no idea what it's doing.

So you phrased a few concepts really eloquently, so let me try to, as a few directions this goes.

So 1, first of all, in the way we write software today, it's not common that we allow it to self-modify.

So your thought is that that is a serious worry if there becomes a reason.

But self-modifying code is available now.

So there's different types of self-modification, right?

There's a personalization, you know, your email app, your Gmail is a self-modifying to you after you log in or whatever.

You can think of it that way, but ultimately it's central, all the information is centralized, but you're thinking of ideas where you're completely, this is a unique entity operating under selective pressures and it changes.

Well, you just, if you think about the fact that our immune systems don't know what's coming at them next, but they have a small set of spanning components.

And if it's a sufficiently expressive system in that any shape or binding region can be approximated with the Lego that is present, then you can have confidence that you don't need to know what's coming at you because the combinatorics are sufficient to reach any configuration needed.

So that's a beautiful, well, terrifying thing to worry about because it's so within our reach.

Whatever I suggest these things, I do always have a concern as to whether or not I will bring them into being by talking about them.

So there's this thing from OpenAI, so next week, I have to talk to the founder of OpenAI.

This idea that their text generation, the new stuff they have for generating text is they didn't want to release it because they're worried about the...

to hear that, but they're going to end up releasing it.

I think talking about it, well at least from my end, I'm more a proponent of technology preventing, so further innovation preventing the detrimental effects of innovation.

Well, we're sort of tumbling down a hill at accelerating speed.

well, I do feel that there are people who've held things back and, you know, died poorer than they might have otherwise been.

I don't think that we should discount the idea that having the smartest people showing off how smart they are by what they've developed may be a terminal process.

I'm very mindful in particular of a beautiful letter that Edward Teller of all people wrote to Leo Zillard where Zillard was trying to figure out how to control the use of atomic weaponry at the end of World War II.

And Teller, rather strangely, because many of us view him as a monster, showed some very advanced moral thinking talking about the slim chance we have for survival and that the only hope is to make war unthinkable.

I do think that not enough of us feel in our gut what it is we are playing with when we are working on technical problems.

And I would recommend to anyone who hasn't seen it, a movie called the Bridge on the River Kwai, about, I believe, captured British POWs who, just in a desire to do a bridge well, end up over collaborating with their Japanese captors.

Well, now you're making me question the unrestricted open discussion of ideas in AI.

I'm just saying that I could make a decent case for either our need to talk about this and to become technologically focused on containing it, or need to stop talking about this and try to hope that the relatively small number of highly adept individuals who

are looking at these problems is small enough that we should in fact be talking about how to contain them.

Well, the way ideas, the way innovation happens, what new ideas develop, Newton with calculus, whether if he was silent, the idea would emerge elsewhere, well in the case of Newton, of course, but in the case of AI, how small is the set of individuals out of which such ideas would arise?

Well, the idea is that the researchers we know and those that we don't know, who may live in countries that don't wish us to know what level they're currently at, are very disciplined in keeping these things to themselves.

Of course, I will point out that there's a religious school in Kerala that developed something very close to the calculus, certainly in terms of infinite series in, I guess, religious prayer and rhyme and prose.

So, you know, it's not that Newton had any ability to hold that back, and I don't really believe that we have an ability to hold it back.

I do think that we could change the proportion of the time we spend worrying about the effects of what if we are successful rather than simply trying to succeed and hope

that we'll be able to contain things later.

So on the idea of intelligence, what form, treading cautiously as we've agreed as we've tumbled down the hill, what form?

So 1 example, Facebook, Google, do want to, I don't know a better word, you want to influence users to behave a certain way.

And so that's 1 kind of example of how intelligence is systems perhaps modifying the behavior of these intelligent human beings in order to sell more product of different kind.

But do you see other examples of this actually emerging in?

Make sure that there's some way in which that there's differential success, heritability,

and variation, and those are the magic ingredients.

And if you really wanted to build a nightmare machine, make sure that the system that expresses the variability has a spanning set so that it can learn to arbitrary levels by making it sufficiently expressive.

So it's your nightmare, but it could also be, it's a really powerful mechanism by which to create, well, powerful systems.

So are you more worried about the negative direction that might go versus the positive?

So you said parasitic, but that doesn't necessarily need to be what the system converges towards.

Parasitism, the dividing line between parasitism and symbiosis is not so clear.

Well, yeah, we could go into that too, but.

think we have to appreciate, you know, are you infected by your own mitochondria?

So, you know, in marriage, you fear the loss of independence, but even though the American therapeutic community may be very concerned about codependence, What's to say that codependence isn't what's necessary to have a stable relationship in which to raise children who are maximally case-selected and require incredible amounts of care because you have to wait 13 years before there's any reproductive payout and most of us don't want our 13-year-olds having kids.

I would say that predators and parasites drive much of our evolution, and I don't know whether to be angry at them or thank them.

Well ultimately, I mean, nobody knows the meaning of life or what even happiness is, but there is some metrics.

They didn't, that's why all the poetry and books are about.

You know, there is some metrics under which you can kind of measure how good it is that these AI systems are roaming about.

So you're more nervous about software than you are optimistic about ideas of, yeah, self-replicating large scale.

I don't think we've really felt where we are.

9-11 was so anomalous compared to everything else we've experienced on American soil that it came to us as a complete shock that that was even a possibility.

What it really was was a highly creative and determined R&D team deep in the bowels of Afghanistan showing us that we had certain exploits that we were open to that nobody had chosen to express.

I can think of several of these things that I don't talk about publicly that just seem to have to do with how relatively unimaginative those who wish to cause havoc and destruction have been up until now.

But the great mystery of our time, of this particular little era, is how remarkably stable we've been since 1945 when we demonstrated the ability to use nuclear weapons in anger.

And we don't know why things like that haven't happened since then.

We've had several close call, We've had mistakes, we've had brinksmanship.

And what's now happened is that we've settled into a sense that, oh, it'll always be nothing.

It's been so long since something was at that level of danger that we've got a wrong idea in our head.

And that's why when I went on the Ben Shapiro show, I talked about the need to resume above ground testing of nuclear devices, because we have people whose developmental experience suggests that when, let's say Donald Trump and North Korea engage on Twitter, oh it's nothing, it's just posturing.

There's a sense that people are in a video game mode which has been the right call since 1945.

So you're worried about a generation which has not seen any existential.

I don't know if, and again, you came from Moscow.

There was a TV show called The Day After that had a huge effect on a generation growing up in the US.

And it talked about what life would be like after a nuclear exchange.

We have not gone through an embodied experience collectively where we've thought about this, and I think it's 1 of the most irresponsible things that the elders among us have done, which is to provide this beautiful garden in which the thorns are cut off of the rose bushes and all of the edges are rounded and sanded.

And so people have developed this totally unreal idea which is everything's going to be just fine.

And do I think that my leading concern is AGI or my leading concern is thermonuclear exchange or gene drives or any 1 of these things, I don't know.

But I know that our time here in this very long experiment here is finite because the toys that we've built are so impressive.

And the wisdom to accompany them has not materialized.

And I think we actually got a wisdom uptick since 1945.

We had a lot of dangerous, skilled players on the world stage who nevertheless, no matter how bad they were, managed to not embroil us in something that we couldn't come back from.

Yeah, and the distance from the Cold War.

You know, I'm very mindful of, there was a Russian tradition, actually, of on your wedding day, going to visit a memorial to those who gave their lives.

Where on the happiest day of your life, you go and you pay homage to the people who fought and died in the Battle of Stalingrad.

I'm not a huge fan of communism, I gotta say, but there were a couple of things that the Russians did that were really positive in the Soviet era.

And I think trying to let people know how serious life actually is, the Russian model of seriousness is better than the American model.

there was a small echo of that after 9-11.

We talk about 9-11, but it's 9-12 that really moved the needle.

When we were all just there and nobody wanted to speak.

We witnessed something super serious and we didn't want to run to our computers and blast out our deep thoughts and our feelings.

And it was profound because we woke up briefly there.

I talk about the gated institutional narrative that sort of programs our lives.

1 of which was the election of Donald Trump.

Another time was the fall of Lehman Brothers when everybody who knew that Bear Stearns wasn't that important knew that Lehman Brothers met AIG was next.

And so If I'm 53 years old and I only remember 3 times that the global narrative was really interrupted, that tells you how much we've been on top of developing events.

I mean, we had the Murrow Federal Building explosion, but it didn't cause the narrative to break.

Around 9-12, we started to wake up out of our slumber.

powers that be did not want to coming together.

They, you know, the admonition was go shopping.

The powers that be was what is that force as opposed to blaming individuals?

Whatever that force is, there's a component

of it that's emergent and there's a component of it that's deliberate.

So give yourself a portfolio with 2 components.

Some amount of it is emergent, but some amount of it is also an understanding that if people come together, they become an incredible force.

And what you're seeing right now, I think, is there are forces that are trying to come together, and There are forces that are trying to come together and there are forces that are trying to push things apart.

And 1 of them is the globalist narrative versus the national narrative, where to the globalist perspective, the nations are bad things in essence, that they're temporary, they're nationalistic, they're jingoistic, it's all negative.

To people in the national, more in the national idiom, they're saying, look, this is where I pay my taxes, This is where I do my army service.

Who the hell are you to tell me that because you've moved into some place that you can make money globally, that you've chosen to abandon other people to whom you have a special and elevated duty?

See all Lex Fridman transcripts on Youtube

Eric Weinstein: Revolutionary Ideas in Science, Math, and Society | Lex Fridman Podcast #16