Nick Bostrom is a philosopher at University of Oxford and the director of the Future of Humanity Institute. He has worked on fascinating and important ideas in existential risks, simulation hypothesis, human enhancement ethics, and the risks of superintelligent AI systems, including in his book Superintelligence. I can see talking to Nick multiple times on this podcast, many hours each time, but we have to start somewhere.

Support this podcast by signing up with these sponsors:
- ExpressVPN at https://www.expressvpn.com/lexpod
- MasterClass: https://masterclass.com/lex
- Cash App - use code "LexPodcast" and download:
- Cash App (App Store): https://apple.co/2sPrUHe
- Cash App (Google Play): https://bit.ly/2MlvP5w

EPISODE LINKS:
Nick's website: https://nickbostrom.com/
Future of Humanity Institute:
- https://twitter.com/fhioxford
- https://www.fhi.ox.ac.uk/
Books:
- Superintelligence: https://amzn.to/2JckX83
Wikipedia:
- https://en.wikipedia.org/wiki/Simulation_hypothesis
- https://en.wikipedia.org/wiki/Principle_of_indifference
- https://en.wikipedia.org/wiki/Doomsday_argument
- https://en.wikipedia.org/wiki/Global_catastrophic_risk

PODCAST INFO:
Podcast website:
https://lexfridman.com/podcast
Apple Podcasts:
https://apple.co/2lwqZIr
Spotify:
https://spoti.fi/2nEwCF8
RSS:
https://lexfridman.com/feed/podcast/
Full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
2:48 - Simulation hypothesis and simulation argument
12:17 - Technologically mature civilizations
15:30 - Case 1: if something kills all possible civilizations
19:08 - Case 2: if we lose interest in creating simulations
22:03 - Consciousness
26:27 - Immersive worlds
28:50 - Experience machine
41:10 - Intelligence and consciousness
48:58 - Weighing probabilities of the simulation argument 
1:01:43 - Elaborating on Joe Rogan conversation
1:05:53 - Doomsday argument and anthropic reasoning
1:23:02 - Elon Musk
1:25:26 - What's outside the simulation?
1:29:52 - Superintelligence
1:47:27 - AGI utopia
1:52:41 - Meaning of life

CONNECT:
- Subscribe to this YouTube channel
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/LexFridmanPage
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

The following is a conversation with Nick Bostrom, a philosopher at University of Oxford and the director of the Future of Humanity Institute.

He has worked on fascinating and important ideas in existential risk, simulation hypothesis, human enhancement ethics, and the risks of superintelligent AI systems, including in his book, Superintelligence.

I can see talking to Nick multiple times on this podcast, many hours each time, because he has done some incredible work in artificial intelligence, in technology space, science, and really philosophy in general.

This conversation was recorded before the outbreak of the coronavirus pandemic that both Nick and I, I'm sure, will have a lot to say about next time we speak.

And perhaps that is for the best, because the deepest lessons can be learned only in retrospect, when the storm has passed.

I do recommend you read many of his papers on the topic of existential risk, including the technical report titled Global Catastrophic Risks Survey that he co-authored with Anders Sandberg.

For everyone feeling the medical, psychological, and financial burden of this crisis, I'm sending love your way.

This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube, review it with 5 stars on Apple Podcast, support it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D-M-A-N.

As usual, I'll do 1 or 2 minutes of ads now, and never any ads in the middle that can break the flow of the conversation.

I hope that works for you and doesn't hurt the listening experience.

This show is presented by Cash App, the number 1 finance app in the App Store.

Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1.

Since Cash App does fractional share trading, let me mention that the order execution algorithm that works behind the scenes to create the abstraction of fractional orders is an algorithmic marvel.

So big props to the Cash App engineers for solving a hard problem that in the end provides an easy interface that takes a step up to the next layer of abstraction over the stock market, making trading more accessible for new investors and diversification much easier.

So, again, if you get Cash App from the App Store or Google Play, and use the code LEX PODCAST, you get $10, and Cash App will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young people around the world.

And now, here's my conversation with Nick Bostrom.

At the risk of asking the Beatles to play Yesterday or the Rolling Stones to play Satisfaction, let me ask you the basics.

That we are living in a computer simulation.

How are we supposed to even think about that?

Well, so the hypothesis is meant to be understood in a literal sense, not that we can kind of metaphorically view the universe as an information processing physical system, but that there is some advanced civilization who built a lot of computers and that what we experience is an effect of what's going on inside 1 of those computers so that the world around us, our own brains, everything we see and perceive and think and feel would exist because this computer is running certain programs.

So do you think of this computer as something similar to the computers of today?

These deterministic sort of Turing machine type things?

Is that what we're supposed to imagine or we're supposed to think of something more like a quantum mechanical system, something much bigger, something much more complicated, something much more mysterious from our current perspective.

You'd need more memory and more processing power.

I don't think anything else would be required.

It might well be that they do have addition, maybe they have quantum computers and other things that would give them even more oomph.

It's kind of plausible, but I don't think it's a necessary assumption in order to get to the conclusion that a technologically mature civilization would be able to create these kinds of computer simulations with conscious beings inside them.

So do you think the simulation hypothesis is an idea that's most useful in philosophy, computer science, physics, sort of, where do you see it having valuable kind of starting point in terms of a thought experiment of it.

I guess it's more informative and interesting and maybe important.

It's not designed to be useful for something else.

But is it philosophically interesting or is there some kind of implications of computer science and physics?

I think not so much for computer science or physics per se.

Certainly it would be of interest in philosophy, I think also to say cosmology or physics in as much as you're interested in the fundamental building blocks of the world and the rules that govern it.

If we are in a simulation, there is then the possibility that, say, physics at the level where the computer running the simulation could be different from the physics governing phenomena in the simulation.

So I think it might be interesting from the point of view of religion or just for kind of trying to figure out what the heck is going on.

So we mentioned the simulation hypothesis so far.

Now, there is also the simulation argument, which I tend to make a distinction.

So simulation hypothesis, we are living in a computer simulation.

Simulation argument, this argument that tries to show that 1 of 3 propositions is true, 1 of which is the simulation hypothesis, but there are 2 alternatives in the original simulation argument, which we can get to.

By the way, confusing terms because people will, I think, probably naturally think simulation argument equals simulation hypothesis, just terminology wise.

So simulation hypothesis means that we are living in a simulation.

The hypothesis that we're living in simulation, simulation argument has these 3 complete possibilities that cover all possibilities.

Although it doesn't on its own tell us which 1.

So the first 1 is that almost all civilizations at our current stage of technological development go extinct before they reach technological maturity.

So there is some great filter that makes it so that basically none of the civilizations throughout, you know, maybe a vast cosmos will ever get to realize the full potential of technological development.

And this could be, theoretically speaking, this could be because most civilizations kill themselves too eagerly or destroy themselves too eagerly, or it might be super difficult to build a simulation.

Now, I think it looks like we would technologically be able to get there in a time span that is short compared to, say, the lifetime of planets and other sort of astronomical

So your intuition is that the build simulation is not...

Well, so there's this interesting concept of technological maturity.

It's kind of an interesting concept to have for other purposes as well.

We can see, even based on our current limited understanding, what some lower bound would be on the capabilities that you could realize by just developing technologies that we already see are possible.

So, for example, 1 of my research fellows here, Eric Drexler, back in the 80s, studied molecular manufacturing.

That is, you could analyze using theoretical tools and computer modeling the performance of various molecularly precise structures that we didn't then and still don't today have the ability to actually fabricate.

But you could say that, well, if we could put these atoms together in this way, then the system would be stable and it would, you know, rotate at this speed and have these computational characteristics.

And he also outlined some pathways that would enable us to get to this kind of molecularly manufacturing in the fullness of time.

You could look at the speed at which, say, it would be possible to colonize the galaxy if you had mature technology.

We have an upper limit, which is the speed of light.

We have a lower current limit, which is how fast current rockets go.

We know we can go faster than that by just making them bigger and have more fuel and stuff.

And you can then start to describe the technological affordances that would exist once a civilization has had enough time to develop, at least those technologies we already know are possible.

Then maybe they would discover other new physical phenomena as well that we haven't realized that would enable them to do even more, but at least there is this kind of basic set of capabilities.

How do we jump from molecular manufacturing to deep space exploration to mature technology, like what's the connection there?

Well, so these would be 2 examples of technological capability sets that we can have a high degree of confidence are physically possible in our universe and that a civilization that was allowed to continue to develop its science and technology would eventually attain.

You can intuit like, we can kind of see the set of breakthroughs that are likely to happen.

So you can see like, what did you call it?

1 is we could just imagine bigger computers using exactly the same parts that we have, so you can kind of scale things that way, right?

But you could also make processors a bit faster if you had this molecular nanotechnology that Erik Drexler described.

He characterized a kind of crude computer built with these parts that would perform at a million times the human brain while being significantly smaller, the size of a sugar cube.

And he made no claim that that's the optimum computing structure.

Like, for what you know, you could build faster computers that would be more efficient.

But at least you could do that if you had the ability to do things that were atomically precise.

You could have this kind of nanomolecular ability to build things atom by atom, and then say at this as a spatial scale that would be attainable through space colonizing technology.

You could then start, for example, to characterize a lower bound on the amount of computing power that a technologically mature civilization would have if it could grab resources, you know, planets and so forth, and then use this molecular nanotechnology to optimize them for computing.

You'd get a very, very high lower bound on the amount of compute.

So technologically mature civilization is 1 that took that piece of technology to its lower bound.

What is a technologically mature civilization?

so that means it's a stronger concept than we really need for the simulation hypothesis.

I just think it's interesting in its own right.

So it would be the idea that there is some stage of technological development where you've basically maxed out, that you developed all those general purpose, widely useful technologies that could be developed, or at least kind of come very close to the 99.9% there or something.

You can think either that there is such a ceiling or you might think it just goes, the technology tree just goes on forever.

I would guess that there is a maximum that you would start to asymptote towards.

So new things won't keep springing up, new ceilings.

In terms of basic technological capabilities, I think that, yeah, there is like a finite set of those that can exist in this universe.

Moreover, I mean, I wouldn't be that surprised if we actually reached close to that level fairly shortly after we have, say, machine superintelligence.

So I don't think it would take millions of years for a human originating civilization to begin to do this.

I think it's like more likely to happen on historical timescales.

But that's an independent speculation from the simulation argument.

I mean, for the purpose of the simulation argument, it doesn't really matter whether it goes indefinitely far up or whether there's a ceiling, as long as we know we can at least get to a certain level.

And it also doesn't matter whether that's going to happen in a hundred years or 5,000 years or 50 million years.

Like the time scales really don't make any difference for this.

Like there's a big difference between a hundred years and 10 million years.

Because you just said, does it matter if we jump scales to beyond historical scales?

So for the simulation argument, sort of, doesn't it matter that we, if it takes 10 million years, it gives us a lot more opportunity to destroy civilization in the meantime.

Yeah, well, So it would shift around the probabilities between these 3 alternatives.

That is, if we are very, very far away from being able to create these simulations, if it's like say the billions of years into the future, then it's more likely that we will fail ever to get there.

There's more time for us to kind of go extinct along the way.

So it is important to think about how hard it is to build a simulation.

In terms of figuring out which of the disjuncts.

But for the simulation argument itself, which is agnostic as to which of these 3 alternatives is true.

It's like, you don't have to, like the simulation argument would be true whether or not we thought this could be done in 500 years or it would take 500 million years.

I mean, I'm sure there might be some people who oppose it, but it doesn't matter.

I mean, it's very nice those 3 cases cover it, but the fun part is at least not saying what the probabilities are, but kind of thinking about, kind of intuiting reasoning about like what's more likely, what are the kinds of things that would make some of the arguments less and more so like, but let's actually, I don't think we went through them.

So number 1 is we destroy ourselves before we ever create simulated.

Right, So that's kind of sad, but we have to think not just what might destroy us.

I mean, so there could be some whatever disasters or meteors slamming the earth a few years from now that could destroy us, right?

But you'd have to postulate in order for this first disjunct to be true that almost all civilizations throughout the cosmos also failed to reach technological maturity.

And the underlying assumption there is that there is likely a very large number of other intelligent civilizations.

Well, if there are, yeah, then they would virtually all have to succumb in the same way.

I mean, that leads off another, I guess there are a lot of little digressions that

Well, there is a set of basic questions that always come up in conversations with interesting people, like the Fermi paradox.

You could almost define whether a person is interesting, whether at some point the question of the Fermi paradox comes up.

Well, so for what it's worth, it looks to me that the universe is very big.

I mean, in fact, according to the most popular current cosmological theories, infinitely big.

And so then it would follow pretty trivially that it would contain a lot of other civilizations.

If you have some local stochasticity and infinitely many, it's like, you know, infinitely many lumps of matter, 1 next to another, there's kind of random stuff in each 1, then you're going to get all possible outcomes with probability 1 infinitely repeated.

So then certainly there would be a lot of extraterrestrials out there.

Even short of that, if the universe is very big, there might be a finite but large number.

If we were literally the only 1, yeah, then of course, if we went extinct, then all of civilizations at our current stage would have gone extinct before becoming technological material.

So then it kind of becomes trivially true that a very high fraction of those went extinct.

But if we think there are many, I mean, it's interesting because there are certain things that plausibly could kill us, like if you look at existential risks, and it might be a different, like the best answer to what would be most likely to kill us might be a different answer than the best answer to the question, if there is something that kills almost everyone, what would that be?

Because that would have to be some risk factor that was kind of uniform over all possible civilization.

Yeah, so in this, for the sake of this argument, you have to think about not just us, but like every civilization dies out before they create the simulation.

Yeah, or something very close to everybody.

Well, so number 2 is the convergence hypothesis.

That is that maybe like a lot of some of these civilizations do make it through to technological maturity, but out of those who do get there, they all lose interest in creating these simulations.

So they just, they have the capability of doing it, but they choose not to.

Not just a few of them decide not to, but out of a million, maybe not even a single 1 of them would do it.

And I think when you say lose interest, that sounds like unlikely because it's like they get bored or whatever, but it could be so many possibilities within that.

I mean, losing interest could be, it could be anything from it being exceptionally difficult to do to fundamentally changing the sort of the fabric of reality if you do it, ethical concerns, all those kinds of things could be exceptionally strong pressures.

Well, certainly, I mean, yeah, ethical concerns.

I mean, in a sense, that's the first assumption that you get to technological maturity where you would have the ability, using only a tiny fraction of your resources, to create many, many simulations.

So it wouldn't be the case that they would need to spend half of their GDP forever in order to create 1 simulation.

And they had this like difficult debate about whether they should invest half of their GDP for this.

It would more be like, well, if any little fraction of the civilization feels like doing this at any point during maybe their millions of years of existence, then there would be millions of simulations.

But certainly, there could be many conceivable reasons for why there would be this convert, many possible reasons for not running ancestor simulations or other computer simulations, even if you could do so cheaply.

By the way, what's an ancestor simulation?

Well, that would be the type of computer simulation that would contain people like those we think have lived on our planet in the past and like ourselves in terms of the types of experiences they have and where those simulated people are conscious.

So not just simulated in the same sense that a non-player character would be simulated in the current computer game, where it's kind of has like an avatar body and then a very simple mechanism that moves it forward or backwards, but something where the simulated being has a brain, let's say, that's simulated at a sufficient level of granularity that it would have the same subjective experiences as we have.

So where does consciousness fit into this?

Do you think simulation, like is there different ways to think about how this can be simulated?

Do we have to simulate each brain within the larger simulation?

Is it enough to simulate just the brains, just the minds, and not the simulation, not the big U and the universe itself?

Is there different ways to think about this?

Yeah, I guess there is a kind of premise in the simulation argument rolled in from philosophy of mind, that is that it would be possible to create a conscious mind in a computer.

And that what determines whether some system is conscious or not is not like whether it's built from organic biological neurons, but maybe something like what the structure of the computation is that it implements.

So we can discuss that if we want, but I think it would be, for as far as my view, that it would be sufficient, say, if you had a computation that was identical to the computation in the human brain down to the level of neurons.

So if you had a simulation with 100 billion neurons connected in the same way as the human brain and you then roll that forward with the same kind of synaptic weights and so forth, so you actually had the same behavior coming out of this as a human with that brain would have done, then I think that would be conscious.

Now, it's possible you could also generate consciousness without having that detailed assimilation.

There I'm getting more uncertain exactly how much you could simplify or abstract away.

I missed where you're placing consciousness in the second.

Well, so if you are a computationalist, do you think that what creates consciousness is the implementation of a computation.

Some property, emergent property of the computation itself.

But then the question is, what's the class of computations such that when they are run, consciousness emerges?

So if you just have like something that adds 1 plus 1 plus 1 plus 1, like a simple computation, you think maybe that's not going to have any consciousness.

If on the other hand, the computation is 1 like our human brains are performing, where as part of the computation, there is like a global workspace, a sophisticated attention mechanism, there is like self-representations of other cognitive processes and a whole lot of other things, that possibly would be conscious.

And in fact, if it's exactly like ours, I think definitely it would.

But exactly how much less than the full computation that the human brain is performing would be required is a little bit, I think, of an open question.

He asked another interesting question as well, which is, would it be sufficient to just have, say, the brain, or would you need the environment...

...In order to generate the same kind of experiences that we have.

And there is a bunch of stuff we don't know.

I mean, if you look at, say, current virtual reality environments, 1 thing that's clear is that we don't have to simulate all details of them all the time in order for, say, the human player to have the perception that there is a full reality in there.

Procedurally generated, it might only render a Scene when it's actually within the view of the player character.

And so similarly, if this environment that we perceive is simulated, it might be that all of the parts that come into our view are rendered at any given time.

And a lot of aspects that never come into view, say the details of this microphone, I'm talking into exactly what each atom is doing at any given point in time might not be part of the simulation, only a more coarse-grained representation.

me is actually from an engineering perspective why the simulation hypothesis is really interesting to think about is how much, how difficult is it to fake, sort of in a virtual reality context, I don't know if fake is the right word, but to construct a reality that is sufficiently real to us to be immersive in the way that the physical world is.

I think that's actually probably an answerable question of psychology, of computer science, of where's the line where it becomes so immersive that you don't wanna leave that world?

Yeah, or that you don't realize while you're in it that it is a virtual world.

Yours is the more sort of the good question about the realism.

But mine, from my perspective, what's interesting is it doesn't have to be real, but it...

How can we construct a world that we wouldn't want to leave?

Yeah, I mean, I think that might be too low a bar.

I mean, if you think, say, when people first had Pong or something like that, I'm sure there were people who wanted to keep playing it for a long time because it was fun and they wanted to be in this little world.

I'm not sure we would say it's immersive.

I mean, I guess in some sense it is, but like an absorbing activity doesn't even have to be.

That's the, so like, I think that bar is deceivingly high.

So they eventually left, so you can play Pong or StarCraft or whatever more sophisticated games for hours, for months, while the World of Warcraft could be in a big addiction, but eventually they escaped that.

So you mean when it's absorbing enough that you would spend your entire, you would choose to spend your entire life in there.

And then thereby changing the concept of what reality is.

Not because you're fooled, but because you've made that choice.

People might have different preferences regarding that.

Some might, even if you had a perfect virtual reality, might still prefer not to spend the rest of their lives there.

I mean, in philosophy, there's this experience machine, thought experiment.

So Robert Nozick had this thought experiment where you imagine some crazy, super-duper neuroscientists of the future have created a machine that could give you any experience you want if you step in there.

And for the rest of your life, you can kind of pre-program it in different ways.

Whatever you dream, you want to be a great artist, a great lover, have a wonderful life, all of these things, if you step into the experience machine, will be your experiences.

Constantly happy, but you would disconnect from the rest of reality and you would float there in a tank.

And so Nozick thought that most people would choose not to enter the experience machine.

I mean, many might want to go there for a holiday, but they wouldn't want to check out of existence permanently.

And so he thought that was an argument against certain views of value, according to what we value is a function of what we experience.

And because in the experience machine, you can have any experience you want, and yet many people would think that would not be much value.

So therefore, what we value depends on other things than what we experience.

I mean, what about the fact that maybe what we value is the up and down of life, so...

You could have up and downs in the experience machine, right?

But what can't you have in the experience machine?

Well, I mean, that then becomes an interesting question to explore, but for example, real connection with other people, if the experience machine is a solo machine where it's only you, like that's something you wouldn't have there.

You would have this subjective experience that would be like fake people.

But if you gave somebody flowers, that wouldn't be anybody there who actually got happy.

It would just be a little simulation of somebody smiling.

But the simulation would not be the kind of simulation I'm talking about in the simulation argument where the simulated creature is conscious.

See all Lex Fridman transcripts on Youtube

Nick Bostrom: Simulation and Superintelligence | Lex Fridman Podcast #83