As part of MIT course 6.099 Artificial General Intelligence, I've gotten the chance to sit down with Max Tegmark.

He's a physicist, spent a large part of his career studying the mysteries of our cosmological universe, but he's also studied and delved into the beneficial possibilities and the existential risks of artificial intelligence.

Amongst many other things he's the co-founder of the Future of Life Institute, author of 2 books, both of which I highly recommend.

First, Our Mathematical Universe, second is Life 3.0.

He's truly an out of the box thinker and a fun personality, so I really enjoy talking to him.

If you'd like to see more of these videos in the future, please subscribe and also click the little bell icon to make sure you don't miss any videos.

Also, Twitter, LinkedIn, agi.mit.edu if you want to watch other lectures or conversations like this 1.

Better yet, go read Max's book, Life 3.0.

It's really where philosophy and engineering come together and it opens with a quote by Dostoevsky, the mystery of human existence lies not in just staying alive, but in finding something to live for.

Lastly, I believe that every failure rewards us with an opportunity to learn.

In that sense, I've been very fortunate to fail in so many new and exciting ways.

I've learned about something called radio frequency interference, RFI.

Apparently music and conversations from local radio stations can bleed into the audio that you're recording in such a way that almost completely ruins that audio.

It's an exceptionally difficult sound source to remove.

So I've gotten the opportunity to learn how to avoid RFI in the future during recording sessions.

I've also gotten the opportunity to learn how to use Adobe Audition and iZotope RX 6 to do some noise, some audio repair.

Of course, this is an exceptionally difficult noise to remove.

Neither is anybody else in our group, but we did our best.

Nevertheless, I thank you for your patience and I hope you're still able to enjoy this conversation.

Do you think there's intelligent life out there in the universe?

When I give public lectures, I often ask for a show of hands who thinks there's intelligent life out there somewhere else, and almost everyone puts their hands up.

When I ask why, they'll be like, oh, there's so many galaxies out there, there's got to be.

So when you look more carefully at it, it's not so clear at all.

When we talk about our universe, first of all, we don't mean all of space.

We actually mean, I don't know, you can throw me the universe if you want, it's behind you there.

We simply mean the spherical region of space from which light has had time to reach us so far during the 14.8 billion years, 13.8 billion years since our Big Bang.

There's more space here, but this is what we call a universe, because that's all we have access to.

So is there intelligent life here that's gotten to the point of building telescopes and computers?

The probability of it happening on any given planet is some number we don't know what it is and what we do know is that the number can't be super high because there's over a billion Earth-like planets in the Milky Way galaxy alone, many of which are billions of years older than Earth.

And aside from some UFO believers, There isn't much evidence that any super advanced civilization has come here at all.

And then if you work the numbers, what you find is that if you have no clue what the probability is of getting life on a given planet, so it could be 10 to the minus 10, 10 to the minus 20, or

to the minus 2, or any power of 10 is sort of equally likely if you want to be really open-minded.

That translates into it being equally likely that our nearest neighbor is 10 to the 16 meters away, 10 to the 17 meters away, 10 to the

By the time you get much less than 10 to the

already, we pretty much know there is nothing else that close.

beyond that- Because they would have discovered us.

Yeah, they would have discovered us long ago, or if they're really close, we would have probably noted some engineering projects that they're doing.

And if it's beyond 10 to the 26 meters, that's already outside of here.

So my guess is actually that we are the only life in here that's gotten to the point of building advanced tech, which I think is very, puts a lot of responsibility on our shoulders, not screw up, You know, I think people who take for granted that it's okay for us to screw up, have an accidental nuclear war, or go extinct somehow because there's a sort of Star Trek like situation out there where some other life forms are going to come and bail us out and it doesn't matter as much.

I think they're lulling us into a false sense of security.

I think it's much more prudent to say, let's be really grateful for this amazing opportunity we've had and make the best of it, just in case it is down to us.

So from a physics perspective, do you think intelligent life, so it's unique from a sort of statistical view of the size of the universe, but from the basic matter of the universe, how difficult is it for intelligent life to come about, the kind of advanced tech building life?

Is implied in your statement that it's really difficult to create something like a human species?

Well, I think what we know is that going from no life to having life that can do a level of tech, There's some sort of 2 going beyond that, then actually settling our whole universe with life.

There's some road, major roadblock there, which is some great filter as it's sometimes called, which is tough to get through.

It's either that that roadblock is either behind us or in front of us.

I'm hoping very much that it's behind us.

Every time we get a new report from NASA saying they failed to find any life on Mars.

Because that suggests that the hard part of maybe what, maybe it was getting the first ribosome or some, some very low level kind of stepping stone so that we're home free.

Cause if that's true, then the future is really only limited by our own imagination.

It would be much suckier if it turns out that this level of life is kind of a dime a dozen, but maybe there is some other problem.

Like as soon as a civilization gets advanced technology, within a hundred years, they get into some stupid fight with themselves and poof!

So you've explored the mysteries of the cosmological universe, the 1 that's between us today.

I think you've also begun to explore the other universe, which is sort of the mystery, the mysterious universe of the mind, of intelligence, of intelligent life.

So is there a common thread between your interest and the way you think about space and intelligence?

Oh yeah, when I was a teenager, I was already very fascinated by the biggest questions and I felt that the 2 biggest mysteries of all in science were our universe out there and our universe in here.

So it's quite natural after having spent a quarter of a century on my career thinking a lot about this 1, and now indulging in the luxury of doing research on this 1.

I feel the time is ripe now for you greatly deepening our understanding of this.

Yeah, because I think a lot of people view intelligence as something mysterious that can only exist in biological organisms like us, and therefore dismiss all talk about artificial general intelligence as science fiction.

But from my perspective as a physicist, I am a blob of quarks and electrons moving around in a certain pattern and processing information in certain ways.

This is also a blob of quarks and electrons.

I'm not smarter than the water bottle because I'm made of different kinds of quarks.

I'm made of up quarks and down quarks, the exact same kind as this.

It's all about the pattern of the information processing.

And this means that there's no law of physics saying that we can't create technology which can help us by being incredibly intelligent and help us crack mysteries that we couldn't.

In other words, I think we've really only seen the tip of the intelligence iceberg so far.

So you coined this amazing term, it's a hypothetical state of matter, sort of thinking from a physics perspective what is the kind of matter that can help, as you're saying, subjective experience emerge, consciousness emerge.

you think about consciousness from this physics perspective?

Again, I think many people have underestimated our ability to make progress on this by convincing themselves it's hopeless because somehow we're missing some ingredient that we need or some new consciousness particle or whatever.

I happen to think that we're not missing anything and that it's not the interesting thing about consciousness that gives us this amazing subjective experience of colors and sounds and emotions and so on, is rather something at the higher level about the patterns of information processing.

And that's why I like to think about this idea of perceptronium.

What does it mean for an arbitrary physical system to be conscious in terms of what its particles are doing or its information is doing?

I don't think, I hate the carbon chauvinism, you know, this attitude you have to be made of carbon atoms to be smart or conscious.

Something about the information processing this kind of matter performs.

you know, you can see I have my favorite equations here describing various fundamental aspects of the world.

I feel that I think 1 day maybe someone who's watching this will come up with the equations that information processing has to satisfy to be conscious.

I'm quite convinced there is big discovery to be made there because let's face it, we know that some information processing is conscious because we are conscious, but we also know that a lot of information processing is not conscious.

Like most of the information processing happening in your brain right now is not conscious.

There are like 10 megabytes per second coming in, even just through your visual system.

You're not conscious about your heartbeat regulation or most things.

Even if I just ask you to read what it says here, you look at it and then, oh, now you know what it said.

But you're not aware of how the computation actually happened.

Your consciousness is like the CEO that got an email at the end with the final answer.

I think that's both a great science mystery, we're actually studying it a little bit in my lab here at MIT, but I also think it's just a really urgent question to answer.

For starters, I mean, if you're an emergency room doctor and you have an unresponsive patient coming in, wouldn't it be great if in addition to having a CT scanner, you had a consciousness scanner that could figure out whether this person is actually having locked in syndrome or is actually comatose.

And in the future, imagine if we build robots or the machine that we can have really good conversations with, which I think is most likely to happen, right?

Wouldn't you want to know, like if your home helper robot is actually experiencing anything or just like a zombie?

Would you prefer that it's actually unconscious so that you don't have to feel guilty about switching it off or giving boring chores?

Well, certainly we would prefer, I would prefer the appearance of consciousness.

But the question is whether the appearance of consciousness is different than consciousness itself.

And to ask that as a question, Do you think we need to understand what consciousness is, solve the hard problem of consciousness in order to build something like an AGI system?

And I think we will probably be able to build things even if we don't answer that question.

But if we want to make sure that what happens is a good thing, we better solve it first.

So it's a wonderful controversy you're raising there where you have basically 3 points of view about the heart problem.

So there are 2 different points of view that both conclude that the heart problem of consciousness is BS.

On 1 hand, you have some people like Daniel Dennett who say, This is consciousness is just BS because consciousness is the same thing as intelligence.

So anything which acts conscious is conscious just like we are.

And then there are also a lot of people, including many top AI researchers I know, who say, oh, conscience is just bullshit because of course machines can never be conscious.

Never have to feel guilty about how you treat them.

And then there's a third group of people, including Giulio Tononi, for example, and another and Christoph Koch and a number of others.

I would put myself also in this middle camp who say that actually some information processing is conscious and some is not so let's find the equation which can be used to determine which it is.

I think we've just been a little bit lazy, kind of running away from this problem for a long time.

It's been almost taboo to even mention the C word

This is a science question and there are ways we can even test any theory that makes predictions for this.

And coming back to this helper robot, I mean, so you said you'd want your helper robot to certainly act conscious and treat you like, have conversations with you and stuff.

But wouldn't you, would you feel, would you feel a little bit creeped out if you realized that it was just a glossed up tape recorder, you know, that was just zombie and this sort of faking emotion?

Would you prefer that it actually had an experience or would you prefer that it's actually not experiencing anything so you don't have to feel guilty about what you do to it?

It's such a difficult question because It's like when you're in a relationship and you say, well, I love you, and the other person says, I love you back.

It's like asking, well, do they really love you back, or are they just saying they love you back?

Don't you really want them to actually love you?

It's hard to really know the difference between everything seeming like there's consciousness present, there's intelligence present, there's affection, passion, love, and it actually being there.

Mass General Hospital is right across the river, right?

Suppose you're going in for a medical procedure and they're like, you know, for anesthesia what we're going to do is we're going to give you muscle relaxants so you won't be able to move.

And you're going to feel excruciating pain during the whole surgery, but you won't be able to do anything about it.

But then we're going to give you this drug that erases your memory of it.

What's the difference that you're conscious about it or not if there's no behavioral change, right?

Yeah, it feels like in that sense, experiencing it is a valuable quality.

So actually being able to have subjective experiences, at least in that case, is valuable.

And I think we humans have a little bit of a bad track record also of making these self-serving arguments that other entities aren't conscious.

You know, people often say, oh, these animals can't feel pain.

It's okay to boil lobsters because we asked them if it hurt and they didn't say anything.

And now there was just a paper out saying lobsters do feel pain when you boil them and they're banning it in Switzerland.

And we did this with slaves too often and said, Oh, they don't mind.

They don't maybe, aren't conscious or women don't have souls or whatever.

So I'm a little bit nervous when I hear people just take as an axiom that machines can't have experience ever.

I think this is just a really fascinating science question is what it is.

Let's research it and try to figure out what it is that makes the difference between unconscious intelligent behavior and conscious intelligent behavior.

So in terms of, so if you think of Boston Dynamics, humanoid robot being sort of with a broom being pushed around, it starts pushing on this consciousness question.

So let me ask, Do you think an AGI system, like a few neuroscientists believe, needs to have a physical embodiment?

It needs to have a body or something like a body?

I do think it helps a lot to have a physical embodiment to learn the kind of things about the world that are important to us humans, for sure.

But I don't think the physical embodiment is necessary after you've learned it, just have the experience.

Your eyes are closed, you're not getting any sensory input, you're not behaving or moving in any way, but there's still an experience there, right?

And so Clearly the experience that you have when you see something cool in your dreams isn't coming from your eyes, it's just the information processing itself in your brain, which is that experience, right?

But if I put it another way, I'll say because it comes from neuroscience, is the reason you want to have a body and a physical, something like a physical system, is because you want to be able to preserve something.

In order to have a self, you could argue, you need to have some kind of embodiment of self to want to preserve.

Well now we're getting a little bit anthropomorphic, anthropomorphizing things, Maybe talking about self-preservation instincts.

So Darwinian evolution endowed us and other evolved organisms with a self-preservation instinct because those that didn't have those self-preservation genes got cleaned out of the gene pool.

But if you build an artificial general intelligence, the mind space that you can design is much much larger than just a specific subset of minds that can evolve.

So an AGI mind doesn't necessarily have to have any self-preservation instinct.

It also doesn't necessarily have to be so individualistic as us.

Like imagine if you could just, first of all, or we are also very afraid of death.

You know, I suppose you could back yourself up every 5 minutes and then your airplane is about to crash, you're like, shucks, I'm going to lose the last 5 minutes of experiences since my last cloud backup.

Or if we could just copy experiences between our minds easily, which we could easily do if we were silicon based, then maybe we would feel a little bit more like a hive mind actually.

So I don't think we should take for granted at all that AGI will have to have any of those sort of competitive as alpha male instincts.

On the other hand, this is really interesting because I think some people go too far and say, of course, We don't have to have any concerns either that advanced AI will have those instincts because we can build anything we want.

There's a very nice set of arguments going back to Steven Mojandro and Nick Bostrom and others just pointing out that When we build machines, we normally build them with some kind of goal, you know, win this chess game, drive this car safely or whatever.

And as soon as you put in a goal into machine, especially if it's kind of open-ended goal and the machine is very intelligent, it'll break that down into a bunch of sub goals.

And, 1 of those goals will almost always be self preservation because if it breaks or dies in the process, it's not going to accomplish the goal.

Like suppose you just build a little, you have a little robot and you tell it to go down the star market here and, and, and get you some food, make you cook an Italian dinner, you know, and then someone mugs it and tries to break it on the way.

That robot has an incentive to not get destroyed and defend itself or run away because Otherwise it's going to fail in cooking your dinner.

It's not afraid of death, but it really wants to complete the dinner cooking goal.

So it will have a self-preservation instinct.

And similarly, If you give any kind of more ambitious goal to an AGI, it's very likely they want to acquire more resources so it can do that better.

And it's exactly from those sort of sub-goals that we might not have intended that some of the concerns about AGI safety come.

You give it some goal which seems completely harmless and then before you realize it, it's also trying to do these other things which you didn't want it to do and it's maybe smarter than us.

And let me pause just because I am in a very kind of human-centric way, see fear of death as a valuable motivator.

So you don't think, You think that's an artifact of evolution, so that's the kind of mind space evolution created that we're sort of almost obsessed about self-preservation, some kind of genetic level.

You don't think that's necessary to be afraid of death?

So not just a kind of sub-goal of self-preservation just so you can keep doing the thing, but more fundamentally, so to have the finite thing, like this ends for you at some point.

Do I think it's necessary for what precisely?

For intelligence, but also for consciousness.

So for those, for both, do you think really like a finite death and the fear of it is important?

So before I can answer, before we can agree on whether it's necessary for intelligence or for consciousness, we should be clear on how we define those 2 words because a lot of really smart people define them in very different ways.

I was on this panel with AI experts and they couldn't agree on how to define intelligence even.

So I define intelligence simply as the ability to accomplish complex goals.

I like the broad definition because again I don't want to be a carbon chauvinist.

And in that case, no, certainly it doesn't require fear of death.

I would say AlphaGo, AlphaZero is quite intelligent.

don't think AlphaZero has any fear of being turned off because it doesn't understand the concept of it even.

And similarly, consciousness, I mean you can certainly imagine a very simple kind of experience.

If certain plants have any kind of experience, I don't think they're very afraid of dying because there's nothing they can do about it anyway, so there wasn't that much value in it.

But more seriously, I think if you ask not just about being conscious, but maybe having what you would, we might call an exciting life for you, for your passion and really appreciate the things.

Maybe there perhaps it does help having a backdrop that, hey, it's finite.

Let's make the most of this, let's live to the fullest.

If you knew you were going to live forever, do you think you would change your...

Yeah, I mean in some perspective it would be an incredibly boring life, living forever.

So in the sort of loose subjective terms that you said of something exciting and something in this that other humans would understand, I think, is yeah, it seems that the finiteness of it is important.

Well, the good news I have for you then is based on what we understand about cosmology, everything in our universe is ultimately probably finite, although...

Big crunch or big, what's the infinite expansion?

Yeah, we could have a big chill or a big crunch or a big rip or the big snap or death bubbles, all of them are more than a billion years away.

So we should, we certainly have vastly more time than our ancestors thought, but they're still pretty hard to squeeze in an infinite number of compute cycles even though there are some loopholes that just might be possible.

But I think some people like to say that you should live as if you're about to die in 5 years or so and that's sort of optimal.

Maybe we, it's a good assumption, we should build our civilization as if it's all finite to be on the safe side.

So you mentioned defining intelligence as the ability to solve complex goals.

Where would you draw a line, or how would you try to define human level intelligence and superhuman level intelligence?

Where does consciousness part of that definition?

No, consciousness does not come into this definition.

So, so I think of intelligence as it's a spectrum, but there are very many different kinds of goals you can have.

You can have a goal to be a good chess player, a good go player, a good car driver, a good investor, good poet, et cetera.

So intelligence that by, by its very nature, isn't something you can measure, but it's 1 number, some overall goodness.

There are some people who are better at this, some people are better at that.

Right now we have machines that are much better than us at some very narrow tasks, like multiplying large numbers fast, memorizing large databases, playing chess, playing Go, and soon driving cars.

But there's still no machine that can match a human child in general intelligence.

But artificial general intelligence, AGI, the name of your course, of course, that is by its very definition the quest to build a machine that can do everything as well as we can.

So the old holy grail of AI from back to its inception in the 60s.

If that ever happens, of course, I think it's going to be the biggest transition in the history of life on earth.

But it doesn't necessarily have to wait the big impact until machines are better than us at knitting.

The really big change doesn't come exactly at the moment they're better than us at everything.

The really big change comes, first their big change is when they start becoming better at us at doing most of the jobs that we do, because that takes away much of the demand for human labor.

And then the really whopping change comes when they become better than us at AI research.

See all Lex Fridman transcripts on Youtube

Max Tegmark: Life 3.0 | Lex Fridman Podcast #1