Wojciech Zaremba is a co-founder of OpenAI. Please support this podcast by checking out our sponsors:
- Paperspace: https://gradient.run/lex to get $15 credit
- Indeed: https://indeed.com/lex to get $75 credit
- Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium
- Grammarly: https://grammarly.com/lex to get 20% off premium
- Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savings

EPISODE LINKS:
Wojciech's Twitter: https://twitter.com/woj_zaremba
Wojciech's Website: https://wojzaremba.com/
OpenAI's Website: https://openai.com/

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
1:18 - The Fermi paradox
8:20 - Systems of government
10:57 - Life, intelligence, and consciousness
18:10 - GPT language model
20:23 - Engineering consciousness
24:40 - Is there an algorithm for intelligence?
32:02 - Neural networks and deep learning
44:53 - Human reward functions
49:47 - Love is part of the human condition
52:14 - Expanding our circle of empathy
56:19 - Psychedelics and meditation
1:07:46 - Ilya Sutskever
1:15:03 - How does GPT work?
1:24:56 - AI safety
1:31:42 - OpenAI Codex
1:45:15 - Robotics
1:54:32 - Developing self driving cars and robots 
2:05:23 - What is the benchmark for intelligence?
2:08:25 - Will we spend more time in virtual reality?
2:10:39 - AI Friendships
2:20:09 - Sleep
2:22:43 - Generating good ideas
2:29:08 - Advice for young people
2:33:52 - Getting started with machine learning
2:37:05 - What is beauty?
2:40:56 - Death
2:47:44 - Meaning of life

SOCIAL:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Reddit: https://reddit.com/r/lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

The following is a conversation with Wojciech Zaremba, co-founder of OpenAI, which is 1 of the top organizations in the world doing artificial intelligence research and development.

Wojciech is the head of language and cogeneration teams, building and doing research on GitHub Copilot, OpenAI Codecs, and GPT-3, and who knows, 4, 5, 6, N, and N plus 1.

And he also previously led OpenAI's robotic efforts.

These are incredibly exciting projects to me that deeply challenge and expand our understanding of the structure and nature of intelligence.

The 21st century, I think, may very well be remembered for a handful of revolutionary AI systems and their implementations.

GPT, codecs, and applications of language models and transformers in general to the language and visual domains may very well be at the core of these AI systems.

To support this podcast, please check out our sponsors.

This is the Lex Friedman podcast and here is my conversation with Wojciech Saremba.

You mentioned that Sam Altman asked about the Fermi paradox and the people

at OpenAI had really sophisticated, interesting answers.

So that's when you knew this is the right team to be working with.

So let me ask you about the Fermi paradox about aliens.

Why have we not found overwhelming evidence for aliens visiting Earth?

I don't have a conviction in the answer, but rather kind of probabilistic perspective on what might be a, let's say, possible answers.

It's also interesting that the question itself even can touch on the, you know, your typical question of what's the meaning of life?

Because if you assume that like we don't see aliens because they destroy themselves, that kind of upweights the focus on making sure that we won't destroy ourselves.

At the moment, the place where I am actually with my belief, and these things also change over the time, is I think that we might be alone in the universe, which actually makes life more, or let's say consciousness life more kind of valuable.

And that means that we should more appreciate it.

So what's your intuition about our galaxy, our universe?

Is it just sprinkled with graveyards of intelligent civilizations?

Or are we truly is life, intelligent life truly unique?

At the moment, my belief that it is unique, but I would say I could also, you know, there was like some footage released with UFO objects, which makes me actually doubt my own belief.

Yeah, I can tell you 1 crazy answer that I have heard.

So apparently, when you look actually at the limits of computation, you can compute more if the temperature of the universe would drop down.

So 1 of the things that aliens might want to do if they are truly optimizing to maximize amount of compute, which maybe can lead to, or let's say simulations or so, it's instead of wasting current entropy of the universe, because we by living, we are actually somewhat wasting entropy, then you can wait for the universe to cool down such that you have more computation?

But that would be 1 of the reasons why you don't see aliens.

It's also possible to some people say that maybe there is not that much point in actually going to other galaxies if you can go inwards.

So there is no limits of what could be an experience if we could, you know, connect machines to our brains.

While there are still some limits if you want to explore the universe.

Yeah, there could be a lot of ways to go inwards too.

Once you figure out some aspect of physics we haven't figured out yet, maybe you can travel to different dimensions.

I mean, travel in three-dimensional space may not be the most fun kind of travel.

There may be just a huge amount of different ways to travel.

And it doesn't require a spaceship going slowly in 3D space to space time.

It also feels 1 of the problems is that speed of light is low and the universe is vast.

And it seems that actually most likely if we want to travel very far, then we would instead of actually sending spaceships with humans that weigh a lot, we would send something similar to what Uri Miller is working on.

These are like a huge sail, which is at first powered, there is a shot of laser from an air and it can propel it to quarter of speed of light.

And sail itself contains a few grams of equipment.

And that might be the way to actually transport matter through universe.

But then when you think what would it mean for humans, it means that we would need to actually put their 3D printer and 3D print a human on another planet.

I don't know, play them YouTube or let's say, or like a 3D print a huge human right away or maybe a womb or so.

With our current techniques of archeology, If a civilization was born and died long long enough ago on earth we wouldn't be able to tell and so that makes me really sad.

And so I think about earth in that same way.

How can we leave some remnants if we do destroy ourselves?

How can we leave remnants for aliens in the future to discover?

Like Wikipedia and YouTube, do we have it like in a satellite orbiting earth with a hard drive?

Like how do we say, how do we back up human civilization for the good parts or all of it is good parts so that it can be preserved longer than our bodies can?

That's kind of a, It's a difficult question.

It also requires the difficult acceptance of the fact that we may die.

And if we die, we may die suddenly as a civilization.

I think it kind of depends on the cataclysm.

We have observed in other parts of the universe that birds of gamma rays, these are high energy rays of light that actually can apparently kill entire galaxy.

So there might be actually nothing even to, nothing to protect us from it.

I'm also, and I'm looking actually at the past civilization.

So it's like Aztecs or so they disappeared from the surface of the earth.

And the way I'm thinking about it is, you know, that definitely they had some problem that they couldn't solve.

And maybe there was a flood and all of a sudden they couldn't drink, there was no potable water and they all died.

And I think that so far the best solution to such problems is I guess technology.

So I mean if they would know that you can just boil water and then drink it after, then that would save their civilization.

And even now when we look actually at the current pandemic, it seems that once again, actually science comes to rescue and somehow science increases size of the action space.

Yeah, but nature has a vastly larger action space.

Yeah, but still, it might be a good thing for us to keep on increasing action space.

But looking at the destruction of human civilization, perhaps expanding the action space will add actions that are easily acted upon, easily executed, and as a result, destroy us.

I was pondering why actually even we have negative impact on the globe.

Because if you ask every single individual, they would like to have clean air.

They would like healthy planet, but somehow it actually is not the case that as a collective we are not going in this direction.

I think that there exists very powerful system to describe what we value.

It assigns actually monetary values to various activities.

At the moment, the problem in the current system is that there are some things which we value.

So even though we value clean air, or maybe we also value lack of destruction on the internet or so, At the moment, these quantities, you know, companies, corporations can pollute them for free.

So in some sense, I wished, or like, that's I guess purpose of politics, to align the incentive systems.

And we are kind of maybe even moving in this direction.

The first issue is even to be able to measure the things that we value.

Then we can actually assign the monetary value to them.

Yeah, and that's, so it's getting the data and also probably through technology, enabling people to vote and to move money around in a way that is aligned with their values.

And that's very much a technology question.

So like having 1 president and Congress and voting that happens every 4 years or something like that, that's a very outdated idea.

be some technological improvements to that kind of idea.

So I'm thinking from time to time about these topics, but it also feels to me that it's a little bit like it's hard for me to actually make correct predictions.

I extremely trust Sam Altman, our CEO, on these topics.

I'm more on the side of being, I guess, naive hippie.

Well, I think self-doubt and I think hippie implies optimism.

Those 2 things are pretty good way to operate.

I mean, still it is hard for me to actually understand how the politics works or like how this, like exactly how the things would play out.

What do you think is rarest in the universe?

What's hardest to build is another engineering way to ask that.

Which is the thing that's hardest to get to?

explain you my kind of mental model, what I think is needed for life to appear.

So I imagine that at some point there was this primordial soup of amino acids and maybe some proteins in the ocean.

And you know, some proteins were turning into some other proteins through reaction.

And you can almost think about this cycle of what turns into what as there is a graph essentially describing which substance turns into some other substance.

And essentially life means that all the sudden in the graph has been created that cycle such that the same thing keeps on happening over and over again.

That's what is needed for life to happen.

And in some sense, you can think almost that you have this gigantic graph and it needs like a sufficient number of edges for the cycle to appear.

And Then from perspective of intelligence and consciousness, my current intuition is that they might be quite intertwined.

First of all, it might not be that it's a binary thing, that you have intelligence or consciousness.

It seems to be more a continuous component.

If we look, for instance, on the event networks recognizing images, people are able to show that the activations of these networks correlate very strongly with activations in visual cortex of some monkeys.

The same seems to be true about language models.

Also, if you, for instance, look, if you train agent in 3D world, at first, you know, it barely recognizes what is going on.

Over the time, it kind of recognizes foreground from a background.

Over the time, it kind of knows where there is a foot and it just follows it.

Over the time, it actually starts having a 3D perception.

So it is possible, for instance, to look inside of the head of an agent and ask what would it see if it looks to the right.

And the crazy thing is, you know, initially when the agents are barely trained, these predictions are pretty bad.

Over the time they become better and better.

You can still see that if you ask what happens when the head is turned by 360 degrees, for some time they think that the different thing appears.

And then at some stage, they understand actually that the same thing supposed to appear.

So they get like a understanding of 3D structure.

It's also, you know, very likely that they have inside some level of, like a symbolic reasoning, like a particular these symbols for other agents.

So when you look at DOTA agents, they collaborate together and they have some anticipation of if they would win battle, they have some expectations with respect to other agents.

I might be too much anthropomorphizing how the things look for me.

But then the fact that they have a symbol for other agents.

And it makes me believe that at some stage as they, you know, as they are optimizing for skills, they would have also symbol to describe themselves.

This is like a very useful symbol to have.

And this particularity, I would call it like a self-consciousness or self-awareness.

And still, it might be different from the consciousness.

So I guess the way how I'm understanding the word consciousness, let's say the experience of drinking a coffee or let's say experience of being a bat.

That's the meaning of the word consciousness.

Yeah, it feels it might be also somewhat related to memory and recurrent connections.

So it's kind of like if you look at anesthetic drugs, they might be, like they essentially, they disturb brain waves such that maybe memory is not formed.

And So there's a lessening of consciousness when you do that.

So that's 1 way to intuit what is consciousness.

There's also kind of another element here.

It could be that it's this kind of self-awareness module that you described, plus the actual subjective experience is a storytelling module that tells us a story about what we're experiencing.

The crazy thing, so let's say, I mean, in meditation, they teach people not to speak story inside of their head.

And there is also some fraction of population who doesn't have actually narrator.

I know people who don't have a narrator and they have to use external people in order to kind of solve tasks that require internal narrator.

So it seems that it's possible to have that experience without the talk.

What are we talking about when we talk about the internal narrator?

Is that the voice when you're reading a book?

Yeah, I thought that's what you are referring to.

Well, I was referring more on the, like, not an actual voice.

I meant, like, there's some kind of like subjective experience feels like it's fundamentally about storytelling to ourselves.

It feels like, like the feeling is a story that is much, much simpler abstraction than the raw sensory information.

So there feels like it's a very high level of abstraction that is useful for me to feel like entity in this world.

Most useful aspect of it is that because I'm conscious, I think there's an intricate connection to me not wanting to die.

So like, it's a useful hack to really prioritize not dying.

Like Those seem to be somewhat connected.

So I'm telling the story of like it's richly feels like something to be me and the fact that me exists in this world.

So I will just refer maybe to the first part, as you said, about that kind of story of describing who you are.

I was thinking about it even, so, you know, obviously I like thinking about consciousness.

And I'm trying to see analogies of these things in AI, what would it correspond to?

So you know, OpenAI train a model called GPT, which can generate pretty amusing text on arbitrary topic.

And 1 way to control GPT is by putting into prefix at the beginning of the text some information what would be the story about.

You can have even chat with, you know, with GPT by saying that the chat is with Lex or Elon Musk or so.

And GPT would just pretend to be you or Elon Musk or so.

And it almost feels that this story that we give ourselves to describe our life, It's almost like things that you put into context of GPT.

GPT itself hasn't learned actually from experience of single human, but from the experience of humanity.

And in some sense, by providing context, it behaves as the thing that you wanted it to be.

It's interesting that the, you know, people have a stories of who they are.

And as you said, these stories, they help them to operate in the world.

But it's also, you know, interesting, I guess, various people find it out through meditation or so, that there might be some patterns that you have learned when you were a kid that actually are not serving you anymore.

And you also might be thinking that that's who you are and that's actually just a story.

Yeah, so it's a useful hack, but sometimes it gets us into trouble.

You wrote that Stephen Hawking, he tweeted, Stephen Hawking asked what breathes fire into equations, which meant what makes given mathematical equations realize the physics of a universe.

Similarly, I wonder what breathes fire into computation.

Okay, so How do we engineer consciousness?

How do you breathe fire and magic into the machine?

So it seems clear to me that not every computation is conscious.

I mean, you can, let's say, just keep on multiplying 1 matrix over and over again.

So in some sense, the question is, what are the computations which could be conscious?

I mean, So 1 assumption is that it has to do purely with computation, that you can abstract away matter.

Other possibilities, it's very important was the realization of computation that it has to do with some force fields or so, and they bring consciousness.

At the moment, my intuition is that it can be fully abstracted away.

So in case of computation, you can ask yourself, what are the mathematical objects or so that could bring such properties?

So for instance, if we think about the models, AI models, what they truly try to do, or like models like GPT is, you know, they try to predict next word or so.

And this turns out to be equivalent to compressing text.

And because in some sense, compression means that you learn the model of reality and you have just to remember where are your mistakes, the better you are in predicting the...

And in some sense, when we look at our experience, also when you look, for instance, at the car driving, you know in which direction it will go, you are good like in prediction.

And, you know, it might be the case that the consciousness is intertwined with compression.

It might be also the case that self-consciousness has to do with compressor trying to compress itself.

So I was just wondering, what are the objects in mathematics or computer science which are mysterious that could have to do with consciousness.

And then I thought, you know, you see in mathematics, there is something called Gadiel theorem, which means if you have sufficiently complicated mathematical system, it is possible to point the mathematical system back on itself.

In computer science, there is something called Halping problem.

So I thought that if we believe that under assumption that consciousness has to do with compression, then you could imagine that the, as you keep on compressing things, then at some point it actually makes sense for the compressor to compress itself.

And in some sense, you know, the crazy...

So, but do you think if we think of a Turing machine, a universal Turing machine, can that achieve consciousness?

So is there something beyond our traditional definition of computation that's required?

So it's a specific computation and I said this computation has to do with compression.

And the compression itself, maybe other way of putting it is like you're internally creating the model of reality.

In order like, you try insight to simplify reality in order to predict what's gonna happen.

And that also feels somewhat similar to how I think actually about my own conscious experience.

So clearly I don't have access to reality.

The only access to reality is through, you know, cable going to my brain.

And my brain is creating a simulation of reality.

And I have access to the simulation of reality.

Are you by any chance aware of the Hutter Prize, Marcus Hutter?

He made this prize for compression of Wikipedia pages.

1, I think has to be perfect compression, which makes, I think that little quirk makes it much less applicable to the general task of intelligence because it feels like intelligence is always going to be messy.

Like perfect compression feels like it's not the right goal but it's nevertheless a very interesting goal.

So for him, intelligence equals compression.

And so the smaller you make the file, given a large Wikipedia page, the more intelligent the system has to be.

So you can make perfect compression if you store errors.

And I think that actually what he meant is you have algorithm plus errors.

By the way, Hutter is a he was a PhD advisor of Shenlech, who is a DeepMind co-founder.

So there's an interesting, and now he's a DeepMind, there's an interesting network of people.

He's 1 of the people that I think seriously took on the task of what would an AGI system look like?

I think for a longest time, the question of AGI was not taken seriously or rather rigorously.

Like mathematically speaking, what would the model look like?

See all Lex Fridman transcripts on Youtube

Wojciech Zaremba: OpenAI Codex, GPT-3, Robotics, and the Future of AI | Lex Fridman Podcast #215