Stephen Wolfram is a computer scientist, mathematician, theoretical physicist, and the founder of Wolfram Research, a company behind Wolfram|Alpha, Wolfram Language, and the Wolfram Physics and Metamathematics projects. Please support this podcast by checking out our sponsors:
- MasterClass: https://masterclass.com/lex to get 15% off
- BetterHelp: https://betterhelp.com/lex to get 10% off
- InsideTracker: https://insidetracker.com/lex to get 20% off

EPISODE LINKS:
Stephen's Twitter: https://twitter.com/stephen_wolfram
Stephen's Blog: https://writings.stephenwolfram.com
Wolfram|Alpha: https://www.wolframalpha.com
A New Kind of Science (book): https://amzn.to/30XoEun
Fundamental Theory of Physics (book): https://amzn.to/30XbAoT
Blog posts:
A 50-Year Quest: https://bit.ly/3NQbZ2P
What Is ChatGPT doing: https://bit.ly/3VOwtuz

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
1:33 - WolframAlpha and ChatGPT
21:14 - Computation and nature of reality
48:06 - How ChatGPT works
1:47:48 - Human and animal cognition
2:01:07 - Dangers of AI
2:09:27 - Nature of truth
2:30:49 - Future of education
3:06:51 - Consciousness
3:15:50 - Second Law of Thermodynamics
3:39:23 - Entropy
3:52:23 - Observers in physics
4:09:15 - Mortality

SOCIAL:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Reddit: https://reddit.com/r/lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

I can tell chat GPT, create a piece of code and then just run it on my computer.

And I'm like, that sort of personalizes for me the what could possibly go wrong, so to speak.

Was that exciting or scary, that possibility?

It was a little bit scary actually, because it's kind of like, if you do that, right, what is the sandboxing that you should have?

And that's sort of a, that's a version of that question for the world.

That is, as soon as you put the AIs in charge of things, you know, how much, how many constraints should there be on these systems before you put the AIs in charge of all the weapons and all these different kinds of systems?

Well, here's the fun part about sandboxes is the AI knows about them.

The following is a conversation with Stephen Wolfram, his fourth time on this podcast.

He's a computer scientist, mathematician, theoretical physicist, and the founder of Wolfram Research, a company behind Mathematica, Wolfram Alpha, Wolfram Language, and the Wolfram Physics and Metamathematics projects.

He has been a pioneer in exploring the computational nature of reality.

And so he's the perfect person to explore with together the new quickly evolving landscape of large language models as human civilization journeys towards building super intelligent AGI.

To support it, please check out our sponsors in the description.

And now, dear friends, here's Stephen Wolfram.

You've announced the integration of Chad GPT and Wolfram Alpha and Wolfram Language.

What are the key differences from the high philosophical level, maybe the technical level, between the capabilities of, broadly speaking, the 2 kinds of systems, large language models and this computational, gigantic computational system infrastructure that is Wolfram Alpha.

Yeah, so what does something like ChatGPT do?

It's mostly focused on make language like the language that humans have made and put on the web and so on.

So, you know, it's primary sort of underlying technical thing is you've given a prompt, it's trying to continue that prompt in a way that's somehow typical of what it's seen based on a trillion words of text that humans have written on the web.

And the way it's doing that is with something which is probably quite similar to the way we humans do the first stages of that using a neural net and so on.

And just saying, given this piece of text, let's ripple through the neural net 1 word and get 1 word at a time of output.

And it's kind of a shallow computation on a large amount of kind of training data that is what we humans have put on the web.

That's a different thing from sort of the computational stack that I spent the last, I don't know, 40 years or so building, which has to do with what can you compute many steps, potentially a very deep computation.

It's not sort of taking the statistics of what we humans have produced and trying to continue things based on that statistics.

Instead, it's trying to take kind of the formal structure that we've created in our civilization, whether it's from mathematics or whether it's from kind of systematic knowledge of all kinds and use that to do arbitrarily deep computations, to figure out things that aren't just, let's match what's already been kind of said on the web, but let's potentially be able to compute something new and different that's never been computed before.

So as a practical matter, our goal is to have made as much as possible of the world computable in the sense that if there's a question that in principle is answerable from some sort of expert knowledge that's been accumulated, we can compute the answer to that question And we can do it in a sort of reliable way that's the best 1 can do given what the expertise that our civilization has accumulated.

It's a much more sort of labor intensive on the side of kind of being creating kind of the computational system to do that.

Obviously, in the kind of the chat GPT world, it's like take things which were produced for quite other purposes, namely all the things we've written out on the web and so on, and sort of forage from that things which are like what's been written on the web.

So I think, as a practical point of view, I view the chat GPT thing as being wide and shallow, and what we're trying to do with building out computation as being this sort of deep, also broad, but most importantly kind of deep type of thing.

I think another way to think about this is you go back in human history, I don't know, a thousand years or something, and you say, what's the typical person going to figure out?

Well, the answer is there's certain kinds of things that we humans can quickly figure out.

That's sort of what our neural architecture and the kinds of things we learn in our lives let us do.

But then there's this whole layer of kind of formalization that got developed in which is, you know, the kind of whole sort of story of intellectual history and whole kind of depth of learning.

That formalization turned into things like logic, mathematics, science, and so on.

And that's the kind of thing that allows 1 to kind of build these towers of, sort of towers of things you work out.

It's not just, I can immediately figure this out.

It's no, I can use this kind of formalism to go step by step and work out something which was not immediately obvious to me.

And that's kind of the story of what we're trying to do computationally is to be able to build those kind of tall towers of what implies what implies what and so on.

And as opposed to kind of the, yes, I can immediately figure it out.

It's just like what I saw somewhere else in something that I heard or remembered or something like this.

What can you say about the kind of formal structure, the kind of formal foundation you can build such a formal structure on, about the kinds of things you would start on in order to build this kind of deep computable knowledge trees.

So the question is sort of how do you think about computation?

1 is what computation intrinsically is like, and the other is what aspects of computation we humans with our minds and with the kinds of things we've learned can sort of relate to in that computational universe.

So if we start on the kind of what can computation be like, it's something I've spent some big chunk of my life studying is imagine that you're, you know, we usually write programs where we kind of know what we want the program to do, and we carefully write, you know, many lines of code, and we hope that the program does what we intended it to do.

But the thing I've been interested in is if you just look at the kind of natural science of programs, so you just say, I'm going to make this program, it's a really tiny program, maybe I even pick the pieces of the program at random, but it's really tiny.

And by really tiny, I mean, you know, less than a line of code type thing.

And you run it, and big discovery that I made in the early 80s is that even extremely simple programs, when you run them, can do really complicated things.

It took me several years to kind of realize that that was a thing, so to speak.

But that realization that even very simple programs can do incredibly complicated things that we very much don't expect.

That discovery, I mean, I realized that that's very much, I think, how nature works.

That is, nature has simple rules, but yet does all sorts of complicated things that we might not expect.

You know, a big thing of the last few years has been understanding that that's how the whole universe and physics works, but that's a quite separate topic.

But so there's this whole world of programs and what they do, and very rich, sophisticated things that these programs can do.

But when we look at many of these programs, we look at them and say, well, that's kind of, I don't really know what that's doing.

So on the 1 hand, we have sort of what's possible in the computational universe.

On the other hand, we have the kinds of things that we humans think about, the kinds of things that are developed in kind of our intellectual history.

And that's really the challenge to sort of making things computational is to connect what's computationally possible out in the computational universe with the things that we humans sort of typically think about with our minds.

Now, that's a complicated kind of moving target because the things that we think about change over time.

We've invented various kinds of ideas and structures and so on.

We're kind of gradually colonizing more and more of this kind of intellectual space of possibilities.

But the real thing, the real challenge is how do you take what is computationally possible, How do you take, how do you encapsulate the kinds of things that we think about in a way that kind of plugs in to what's computationally possible?

And actually the big sort of idea there is this idea of kind of symbolic programming, symbolic representations of things.

And so the question is, when you look at sort of everything in the world and you kind of, you know, you take some visual scene or something you're looking at, and you say, well, how do I turn that into something that I can kind of stuff into my mind?

You know, there are lots of pixels in my visual scene, but the things that I remembered from that visual scene are, you know, there's a chair in this place.

It's a kind of a symbolic representation of the visual scene.

There are 2 chairs and a table or something, rather than there are all these pixels arranged in all these detailed ways.

And so the question then is how do you take sort of all the things in the world and make some kind of representation that corresponds to the types of ways that we think about things?

And human language is sort of 1 form of representation that we have.

We talk about chairs, that's a word in human language and so on.

How do we take, but human language is not in and of itself something that plugs in very well to sort of computation.

It's not something from which you can immediately compute consequences and so on.

And so you have to kind of find a way to take sort of the stuff we understand from human language and make it more precise.

And that's really the story of symbolic programming.

And what that turns into is something which I didn't know at the time it was going to work as well as it has.

But back in the 1979 or so, I was trying to build my first big computer system and trying to figure out, how should I represent computations at a high level?

And I kind of invented this idea of using kind of symbolic expressions, you know, structured as it's kind of like a function and a bunch of arguments, but that function doesn't necessarily evaluate to anything, it's just a thing that sits there representing a structure.

And so building up that structure, and it's turned out that structure has been extremely, it's a good match for the way that we humans, it seems to be a good match for the way that we humans kind of conceptualize higher level things.

And it's been for the last, I don't know, 45 years or something, it's served me remarkably well.

So building up that structure using this kind of symbolic representation.

But what can you say about abstractions here?

Because you could just start with your physics project, you could start at a hypergraph at a very, very low level and build up everything from there, but you don't.

You take the highest level of abstraction, convert that, the kind of abstraction that's convertible to something computable using symbolic representation, and then that's your new foundation for that little piece of knowledge.

Right, so the sort of a very important phenomenon that is kind of a thing that I've sort of realized is just, it's 1 of these things that sort of in the future of kind of everything is going to become more and more important is this phenomenon of computational irreducibility.

And the question is, if you know the rules for something, you have a program, you're gonna run it, you might say, I know the rules, great, I know everything about what's gonna happen.

Well, in principle, you do, because you can just run those rules out and just see what they do.

You might run them a million steps, you see what happens, et cetera.

The question is, can you like immediately jump ahead and say, I know what's gonna happen after a million steps and the answer is 13 or something.

And 1 of the very critical things to realize is, if you could reduce that computation, there is in a sense no point in doing the computation.

The place where you really get value out of doing the computation is when you had to do the computation to find out the answer.

But this phenomenon that you have to do the computation to find out the answer, this phenomenon of computational irreducibility seems to be tremendously important for thinking about lots of kinds of things.

So 1 of the things that happens is, okay, you've got a model of the universe at the low level in terms of atoms of space and hypergraphs and rewriting hypergraphs and so on.

And it's happening 10 to the 100 times every second, let's say.

Well, you say, great, then we've nailed it.

Well, the problem is the universe can figure out what it's gonna do.

It does those 10 to the 100 steps, but for us to work out what it's gonna do, we have no way to reduce that computation.

The only way to do the computation, to see the result of the computation, is to do it.

And if we're operating within the universe, we're kind of, there's no opportunity to do that because the universe is doing it as fast as the universe can do it.

So what we're trying to do, and a lot of the story of science, a lot of other kinds of things, is finding pockets of reducibility.

That is, you could have a situation where everything in the world is full of computational irreducibility.

The only way we can figure out what's gonna happen next is just let the system run and see what happens.

So In a sense, the story of most kinds of science, inventions, a lot of kinds of things, is the story of finding these places where we can locally jump ahead.

And 1 of the features of computational reducibility is there are always pockets of reducibility.

There are always places, there are always an infinite number of places where you can jump ahead.

There's no way where you can jump completely ahead, but there are little patches, little places where you can jump ahead a bit.

And I think, we can talk about physics project and so on, but I think the thing we realize is, we kind of exist in a slice of all the possible computational irreducibility in the universe.

We exist in a slice where there's a reasonable amount of predictability.

And in a sense, as we try and construct these kind of higher levels of abstraction, symbolic representations and so on.

What we're doing is we're finding these lumps of reducibility that we can kind of attach ourselves to and about which we can kind of have fairly simple narrative things to say.

Because in principle, you know, I say, what's gonna happen in the next few seconds?

Oh, there are these molecules moving around in the air in this room, and oh gosh, it's an incredibly complicated story, and that's a whole computationally irreducible thing, most of which I don't care about, And most of it is, well, the air's still gonna be here and nothing much is going to be different about it.

And that's a kind of reducible fact about what is ultimately at an underlying level of computationally irreducible process.

And life would not be possible if we didn't have a large number of such reducible pockets.

Pockets amenable to reduction into something symbolic.

I mean, life in the way that we experience it, that, I mean, 1 might, depending on what we mean by life, so to speak, the experience that we have of sort of consistent things happening in the world, the idea of space, for example, where there's, you know, we can just say, you're here, you move there.

It's still you in that different place, even though you're made of different atoms of space and so on.

This is this idea that it's, that there's sort of this level of predictability of what's going on, that's us finding a slice of reducibility in what is underneath this computational irreducible kind of system.

And I think that's sort of the thing which is actually my favorite discovery of the last few years is the realization that it is sort of the interaction between the sort of underlying computational irreducibility and our nature as kind of observers who sort of have to key into computational reducibility.

That fact leads to the main laws of physics that we discovered in the 20th century.

So this is, we talk about this in more detail, but this is, to me, it's kind of our nature as observers, the fact that we are computationally bounded observers, we don't get to follow all those little pieces of computational irreducibility, to stuff what is out there in the world into our minds requires that we are looking at things that are reducible, we are compressing, we're extracting just some essence, some kind of symbolic essence of what's the detail of what's going on in the world, that together with 1 other condition that at first seems sort of trivial but isn't, which is that we believe we are persistent in time.

Here's the thing, at every moment, according to our theory, we're made of different atoms of space.

At every moment, sort of the microscopic detail of what the universe is made of is being rewritten.

And that's, and in fact, the very fact that there's coherence between different parts of space is a consequence of the fact that there are all these little processes going on that kind of knit together the structure of space.

It's kind of like if you wanted to have a fluid with a bunch of molecules in it, if those molecules weren't interacting, you wouldn't have this fluid that would pour and do all these kinds of things.

It would just be sort of a free-floating collection of molecules.

So similarly it is with space, that the fact that space is kind of knitted together as a consequence of all this activity in space.

And the fact that what we consist of, sort of this series of, we're continually being rewritten.

And the question is, why is it the case that we think of ourselves as being the same us through time?

I think it's a key aspect of what we see as sort of our consciousness, so to speak, is that we have this kind of consistent thread of experience.

Well, isn't that just another limitation of our mind that we want to reduce reality into some, that kind of temporal consistency is just a nice narrative to tell ourselves.

Well, the fact is, I think it's critical to the way we humans typically operate is that we have a single thread of experience.

If you imagine sort of a mind where you have, maybe that's what's happening in various kinds of minds that aren't working the same way other minds work, is that you're splitting into multiple threads of experience.

It's also something where, you know, when you look at, I don't know, quantum mechanics, for example, in the insides of quantum mechanics, it's splitting into many threads of experience.

But in order for us humans to interact with it, you kind of have to knit all those different threads together so that we say, oh yeah, a definite thing happened, and now the next definite thing happens, and so on.

And I think, you know, sort of inside, it's sort of interesting to try and imagine what's it like to have kind of these fundamentally multiple threads of experience going on.

I mean, right now, different human minds have different threads of experience.

We just have a bunch of minds that are interacting with each other, but we don't have a, you know, within each mind, there's a single thread.

And that's a, that is indeed a simplification.

I think it's a, it's a thing, you know, the general computational system does not have that simplification.

And it's 1 of the things, you know, I, people often seem to think that, you know, consciousness is the highest level of kind of things that can happen in the universe, so to speak, but I think that's not true.

I think it's actually a specialization in which, among other things, you have this idea of a single thread of experience, which is not a general feature of anything that could kind of computationally happen in the universe.

So it's a feature of a computationally limited system that's only able to observe reducible pockets.

So, I mean this word observer, it means something in quantum mechanics, it means something in a lot of places.

It means something to us humans as conscious beings.

So what's the importance of the observer?

What is the observer and what's the importance of the observer in the computational universe?

So this question of what is an observer, what's the general idea of an observer, is actually 1 of my next projects, which got somewhat derailed by the current sort of AI mania.

Is there a connection there, or is that, do you think the observer is primarily a physics phenomena?

So 1 question is, what is a general observer?

So we know, we have an idea, what is a general computational system?

We think about Turing machines, we think about other models of computation.

There's a question, what is a general model of an observer?

And there's kind of observers like us, which is kind of the observers we're interested in.

You know, we could imagine an alien observer that deals with computational irreducibility and it has a mind that's utterly different from ours and completely incoherent with what we're like.

But the fact is that if we are talking about observers like us, that 1 of the key things is this idea of kind of taking all the detail of the world and being able to stuff it into a mind.

Being able to take all the detail and kind of extract out of it a smaller set of kind of degrees of freedom, a smaller number of elements that will sort of fit in our minds.

And I think this question, so I've been interested in trying to characterize what is the general observer.

And the general observer is, I think, in part, there are many, let me give an example of it.

You have a gas, it's got a bunch of molecules bouncing around, And the thing you're measuring about the gas is its pressure.

And the only thing you as an observer care about is pressure.

And that means you have a piston on the side of this box and the piston is being pushed by the gas.

And there are many, many different ways that molecules can hit that piston.

But all that matters is the kind of aggregate of all those molecular impacts, because that's what determines pressure.

So there's a huge number of different configurations of the gas, which are all equivalent.

So I think 1 key aspect of observers is this equivalency of many different configurations of a system, saying all I care about is this aggregate feature, all I care about is this overall thing.

And we see that in lots of different, again, it's the same story over and over again, that There's a lot of detail in the world, but what we are extracting from it is something, a sort of a thin summary of that detail.

I mean, if we look at the observer that's the human mind, it seems like there's a lot of very, as represented by natural language, for example, there's a lot of really crappy approximation.

Right, right, you don't know, it could be the case.

You're just measuring the aggregate impacts of these molecules, but there is some tiny, tiny probability that molecules will arrange themselves in some really funky way, and that just measuring that average isn't going to be the main point.

By the way, an awful lot of science is very confused about this because you look at papers and people are really keen, they draw this curve and they have these bars on the curve and things, it's just this curve.

And it's this 1 thing, and it's supposed to represent some system that has all kinds of details in it.

And this is a way that lots of science has gotten wrong, because people say, I remember years ago I was studying snowflake growth.

You have a snowflake and it's growing, it has all these arms, it's doing complicated things, but there was a literature on this stuff and it talked about what's the rate of snowflake growth.

And it got pretty good answers for the rate of the growth of the snowflake.

And then I looked at it more carefully and they had these nice curves of snowflake growth rates and so on.

I looked at it more carefully and I realized, according to their models, the snowflake will be spherical.

And so they got the growth rate right, but the detail was just utterly wrong.

And not only the detail, the whole thing was not capturing, you know, it was capturing this aspect of the system that was in a sense missing the main point of

Snowflakes start in the phase of water that's relevant to formation of snowflakes, it's a phase of ice which starts with a hexagonal arrangement of water molecules.

And so it starts off growing as a hexagonal plate.

plate, oh, 0, versus sphere, sphere versus plate.

Well, no, no, but it's much more than that.

Typical snowflakes have little dendritic arms.

And what actually happens is, it's kind of cool because you can make these very simple, discreet models with cellular automata and things that figure this out.

You start off with this hexagonal thing, and then the places, it starts to grow little arms, and every time a little piece of ice adds itself to the snowflake, the fact that that ice condensed from the water vapor heats the snowflake up locally, And so it makes it less likely for another piece of ice to accumulate right nearby.

So this leads to a kind of growth inhibition.

So you grow an arm and it is a separated arm because right around the arm it got a little bit hot and it didn't add more ice there.

So what happens is it grows, you have a hexagon, it grows out arms, the arms grow arms, and then the arms grow arms grow arms, and eventually, actually it's kind of cool because it actually fills in another hexagon, a bigger hexagon.

And when I first looked at this, I had a very simple model for this, I realized when it fills in that hexagon, it actually leaves some holes behind.

So I thought, well, is that really right?

So I look at these pictures of snowflakes, and sure enough, they have these little holes in them that are kind of scars of the way that these arms grow out.

And presumably there's a limitation of how

I mean the thing falls through the, I mean I think it hits the ground at some point.

I think you can grow, I think you can grow in the lab, I think you can grow pretty big ones.

I think you can grow many, many iterations of this kind of, goes from hexagon, it grows out arms, it turns back, it fills back into a hexagon, it grows more arms again.

and fluffy is a three-dimensional property, no?

Snow is, okay, so what makes, we're really in a detail here.

There's multiple snowflakes become fluffy.

And what happens is, if you have snow that is just pure hexagons, they can, you know, they fit together pretty well.

It's not, it doesn't make, it doesn't have a lot of air in it, and they can also slide against each other pretty easily, and so the snow can be pretty, you know, can, I think avalanches happen sometimes when the things tend to be these, you know, hexagonal plates and it kind of slides?

But then when the thing has all these arms that have grown out, it's not, they don't fit together very well.

And that's why the snow has lots of air in it.

See all Lex Fridman transcripts on Youtube

Stephen Wolfram: ChatGPT and the Nature of Truth, Reality & Computation | Lex Fridman Podcast #376