Michael Littman is a computer scientist at Brown University. Please support this podcast by checking out our sponsors:
- SimpliSafe: https://simplisafe.com/lex and use code LEX to get a free security camera
- ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free
- MasterClass: https://masterclass.com/lex to get 2 for price of 1
- BetterHelp: https://betterhelp.com/lex to get 10% off

EPISODE LINKS:
Michael's Twitter: https://twitter.com/mlittmancs
Michael's Website: https://www.littmania.com/
Michael's YouTube: https://www.youtube.com/user/mlittman

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
2:30 - Robot and Frank
4:50 - Music
8:01 - Starring in a TurboTax commercial
18:14 - Existential risks of AI
36:36 - Reinforcement learning
1:02:24 - AlphaGo and David Silver
1:12:03 - Will neural networks achieve AGI?
1:24:30 - Bitter Lesson
1:37:20 - Does driving require a theory of mind?
1:46:46 - Book Recommendations
1:52:08 - Meaning of life

CONNECT:
- Subscribe to this YouTube channel
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/LexFridmanPage
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

The following is a conversation with Michael Littman, a computer science professor at Brown University doing research on and teaching machine learning, reinforcement learning, and artificial intelligence.

He enjoys being silly and lighthearted in conversation, so this was definitely a fun 1.

Quick mention of each sponsor, followed by some thoughts related to the episode.

Thank you to SimpliSafe, a home security company I use to monitor and protect my apartment.

ExpressVPN, the VPN I've used for many years to protect my privacy on the internet, Masterclass, online courses that I enjoy from some of the most amazing humans in history, and BetterHelp, online therapy with a licensed professional.

Please check out these sponsors in the description to get a discount and to support this podcast.

As a side note, let me say that I may experiment with doing some solo episodes in the coming month or 2.

The 3 ideas I have floating in my head currently is to use 1.

A book to drive a conversation about a set of related concepts.

For example, I could use 2001, A Space Odyssey, or Ex Machina to talk about AGI for 123 hours.

Or I could do an episode on the, yes, rise and fall of Hitler and Stalin, each in a separate episode, using relevant books and historical moments for reference.

I find the format of a solo episode very uncomfortable and challenging, but that just tells me that it's something I definitely need to do and learn from the experience.

Of course, I hope you come along for the ride.

Also, since we have all this momentum built up on announcements, I'm giving a few lectures on machine learning at MIT this January.

In general, if you have ideas for the episodes, for the lectures, or for just short videos on YouTube, let me know in the comments that I still definitely read despite my better judgment and the wise sage device of the great Joe Rogan.

If you enjoy this thing, subscribe on YouTube, review it with 5 Stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter, Alex Friedman.

And now, here's my conversation with Michael Littman.

I saw a video of you talking to Charles Isbell about Westworld, the TV series.

You guys were doing the kind of thing where you're watching new things together, but let's rewind back.

Is there a sci-fi movie or book or shows that was profound, that had an impact on you philosophically or just like specifically something you enjoyed nerding out about?

I think a lot of us have been inspired by robots in movies.

The 1 that I really like is, there's a movie called Robot and Frank, which I think is really interesting because it's very near-term future where robots are being deployed as helpers in people's homes.

And It was, and we don't know how to make robots like that at this point, but it seemed very plausible.

And I thought that was really cool because they're awkward, they do funny things, it raised some interesting issues, but it seemed like something that would ultimately be helpful and good if we could do it right.

He was an older cranky jewel thief, yeah.

It's kind of a funny little thing, which is, you know, he's a jewel thief, and so he pulls the robot into his life, which is something you could imagine taking a home robotics thing and pulling into whatever quirky thing that's involved in your existence.

Yeah, and I think from that perspective, I mean, not all of us are jewel thieves, and so when we bring our robots into our lives.

It explains a lot about this apartment, actually.

But no, the idea that people should have the ability to make this technology their own, that it becomes part of their lives.

And I think that's, it's hard for us as technologists to make that kind of technology.

It's easier to mold people into what we need them to be.

And just that opposite vision, I think, is really inspiring.

And then there's a anthropomorphization where we project certain things on them.

Because I think the robot was kind of dumb.

But I have a bunch of Roombas I play with, and you immediately project stuff onto them, much greater level of intelligence.

We'll probably do that with each other too.

1 of the things we're learning from AI is where we are smart and where we are not smart.

You also enjoy, as people can see, and I enjoy myself watching you sing and even dance a little bit, a little bit, A little bit of dancing.

A little bit of dancing, that's not quite my thing.

As a method of education or just in life, in general, so easy question.

What's the definitive, objectively speaking, top 3 songs of all time.

Maybe something that, you know, to walk that back a little bit, maybe something that others might be surprised by, the 3 songs that you kind of enjoy.

That is a great question that I cannot answer, but instead let me tell you a story.

I've been watching the presidential debates and vice presidential debates, and it turns out, yeah, it's really, you can just answer any question you want.

I've enjoyed pop music ever since I was very young.

So 60s music, 70s music, 80s music, this is all awesome.

And then I had kids and I think I stopped listening to music and I was starting to realize that the, like my musical taste had sort of frozen out.

And so I decided in 2011, I think, to start listening to the top

So I'd be on the treadmill and I would listen to that week's top 10 songs so I could find out what was popular now.

And what I discovered is that I have no musical taste whatsoever.

And so the first time I'd hear a song, it's the first week that it was on the charts, I'd be like, ugh.

And then the second week, I was into it a little bit.

And by the fourth week, it was like just part of me.

And so I'm afraid that I can't tell you the most, my favorite song of all time because it's whatever I heard most recently.

People have told me that there's an art to listening to music as well.

You can start to, if you listen to a song just carefully, explicitly just force yourself to really listen, you start to, I did this when I was part of a jazz band and fusion band in college, is you start to hear the layers of the instruments.

You start to hear the individual instruments and you start to, you can listen to classical music or to orchestra this way, you can listen to jazz this way.

It's funny to imagine you now to walking that forward to listening to pop hits now as like a scholar.

Listening to like Cardi B or something like that, or Justin Timberlake.

They've both been in the top 10 since I've been listening.

If you haven't heard Justin Timberlake's top 10 in the last few years, there was 1 song that he did where the music video was set at essentially NUR-IPS.

Yeah, yeah, it's like at an academic conference and he's doing a demo.

It was sort of a cross between the Apple, like Steve Jobs kind of talk and NeurIPS.

So it's always fun when AI shows up in pop culture.

I wonder if he consulted somebody for that.

So maybe on that topic, I've seen your celebrity in multiple dimensions, but 1 of them is you've done cameos in different places.

I've seen you in a Turbo Tax commercial as like, I guess, the brilliant Einstein character.

And the point is that TurboTax doesn't need somebody like you.

But yes, they were specifically emphasizing the idea that You don't need to be a computer expert to be able to use their software.

It was an intro computer science class for non-concentrators, non-majors.

And Sometimes when people would visit campus, they would check in to say, hey, we wanna see what a class is like, can we sit on your class?

So a person came to my class who was the daughter of the brother of the Husband of the best friend of my wife.

Anyway, basically a family friend came to campus to check out Brown and asked to come to my class and came with her dad.

Her dad is, who I've known from various kinds of family events and so forth, but he also does advertising.

And he said that he was recruiting scientists for this ad, this TurboTax set of ads.

And he said, we wrote the ad with the idea that we get like the most brilliant researchers, but they all said no.

So can you help us find like B-level scientists?

I'm like, sure, That's who I hang out with, so that should be fine.

So I put together a list and I did what some people called a Dick Cheney.

So I included myself on the list of possible candidates, you know, with a little blurb about each 1 and why I thought that it would make sense for them to do it.

And they reached out to a handful of them, but then they ultimately, they YouTube stalked me a little bit and they thought, oh, I think he could do this.

And they said, okay, we're gonna offer you the commercial.

So it was such an interesting experience because they have another world.

The people who do nationwide kind of ad campaigns and television shows and movies and so forth, it's quite a remarkable system that they have going.

Yeah, so I went to, it was just somebody's house that they rented in New Jersey.

But in the commercial, it's just me and this other woman.

In reality, there were 50 people in that room and another, I don't know, half a dozen kind of spread out around the house in various ways.

There were people whose job it was to control the sun.

They were in the backyard on ladders putting filters up to try to make sure that the sun didn't glare off the window in a way that would wreck the shot.

So there was like 6 people out there doing that.

There was 3 people out there giving snacks, the craft table.

There was another 3 people giving healthy snacks because that was a separate craft table.

There was 1 person whose job it was to keep me from getting lost.

And I think the reason for all this is because so many people are in 1 place at 1 time, they have to be time efficient, they have to get it done.

The morning they were gonna do my commercial, in the afternoon they were gonna do a commercial of a mathematics professor from Princeton, they had to get it done, no wasted time or energy.

And so there's just a fleet of people all working as an organism and it was fascinating.

I was just the whole time, just looking around, like this is so neat.

Like 1 person whose job it was to take the camera off of the cameraman so that someone else whose job it was to remove the film canister because every couple's takes, they had to replace the film because film gets used up.

It was just, I don't know, I was geeking out the whole time.

It looked the opposite, like there was more than 2 people there.

I mean, the person who I was in the scene with is a professional.

And when I got there, they had given me a script, as such as it was.

And then I got there and they said, We're gonna do this as improv.

Like this is not, I don't know what this, I don't know what you're telling me to do here.

I guess I got pulled into the story because like where the heck did you come from?

Like how did you show up in this random person's house?

Yeah, well, I mean, the reality of it is I stood outside in the blazing sun.

There was someone whose job it was to keep an umbrella over me because I started to schwitz, I started to sweat.

And so I would wreck the shot because my face was all shiny with sweat.

So there was 1 person who would dab me off, had an umbrella.

But yeah, like the reality of it, like why is this strange stalkery person hanging around outside somebody's house?

We'll have to look in, we'll have to wait for the book.

But are you, So you make, like you said, YouTube, you make videos yourself, you make awesome parody, sort of parody songs that kind of focus in on a particular aspect of computer science.

How much, those seem really natural, How much production value goes into that?

The videos, almost all the videos, except for the ones that people would have actually seen, were just me.

I generally find a backing track online because I'm like you, can't really play an instrument.

And then I do, in some cases I'll do visuals using just like PowerPoint.

Lots and lots of PowerPoint to make it sort of like an animation.

The most produced 1 is the 1 that people might have seen, which is the overfitting video that I did with Charles Isbell.

And that was produced by the Georgia Tech and Udacity people, because we were doing a class together.

It was kind of, I usually do parody songs kind of to cap off a class at the end of a class.

So that 1 you're wearing, so it was just a thriller.

You're wearing the Michael Jackson, the red leather jacket.

The interesting thing with podcasting that you're also into is that I really enjoy is that there's not a team of people.

It's kind of more, because you know, there's something that happens when there's more people involved than just 1 person, that just the way you start acting, I don't know, there's a censorship, you're not given, especially for like slow thinkers like me, you're not, and I think most of us are, if we're trying to actually think, we're a little bit slow and careful.

It kind of, large teams get in the way of that.

To me, it's very popular to criticize, quote unquote, mainstream media.

But there is legitimacy to criticizing them.

The same, I love listening to NPR, for example, but every, it's clear that there's a team behind it, there's a commercial, there's constant commercial breaks, there's this kind of like rush of like, Okay, I have to interrupt you now because we have to go to commercial.

Just this whole, it creates, it destroys the possibility of nuanced conversation.

Evian, which Charles Isbell, who I talked to yesterday, told me that Evian is naive backwards, which the fact that his mind thinks this way is just quite brilliant.

Anyway, there's a freedom to this podcast.

Awkward, which by the way is a palindrome.

That's a palindrome that I happen to know from other parts of my life.

So what was the most challenging parody song to make?

I wrote the lyrics really quickly, and then I gave it over to the production team.

They recruited a a cappella group to sing.

It's great having a team, because then you can just focus on the part that you really love, which in my case is writing the lyrics.

For me, the most challenging 1, not challenging in a bad way, but challenging in a really fun way, was I did, 1 of the parody songs I did is about the halting problem in computer science, the fact that You can't create a program that can tell for any other arbitrary program whether it's actually going to get stuck in an infinite loop or whether it's going to eventually stop.

And so I did it to an 80s song because I hadn't started my new thing of learning current songs.

And- Sing Me A Song, You're The Piano Man.

So The lyrics are great because first of all, it rhymes.

I've done Rolling Stones songs, which turn out to have no rhyme scheme whatsoever.

They're just sort of yelling and having a good time, which makes it not fun from a parody perspective because you can say anything.

But the lines rhymed and there was a lot of internal rhymes as well.

And so figuring out how to sing with internal rhymes, a proof of the halting problem was really challenging.

What about, last question on this topic, What about the dancing in the thriller video?

They had me in the studio and they gave me the jacket and it's like, well, you can't, if you have the jacket and the glove, like there's not much you can do.

And then they said, why don't you dance a little bit?

There was a scene with me and Charles dancing together.

They did not use it in the video, but we recorded it.

And Charles, who has this beautiful, wonderful voice, doesn't really sing.

And so that was why I designed the song with him doing a spoken section and me doing the singing.

So 1 of the other things Charles said is that everyone knows you as a super nice guy, super passionate about teaching and so on.

What he said, don't know if it's true, that despite the fact that you are a super- Kill the man

It's the Johnny Cash song, kill the man in Reno just to watch him die.

That you actually do have some strong opinions on some topics.

So if this in fact is true, what strong opinions would you say you have?

Is there ideas you think maybe in artificial intelligence, machine learning, maybe in life, that you believe is true that others might, some number of people might disagree with you on.

So I try very hard to see things from multiple perspectives.

great Calvin and Hobbes cartoon where, do you know?

Okay, so Calvin's dad is always kind of a bit of a foil and he talked Calvin into, Calvin had done something wrong.

The dad talks him into like seeing it from another perspective.

And Calvin, like this breaks Calvin because he's like, oh my gosh, now I can see the opposite sides of things.

And so it becomes like a cubist cartoon where there is no front and back, everything's just exposed, and it really freaks him out.

It's like, oh good, no, I can make that go away.

But I'm that, I live in that world where I'm trying to see everything from every perspective all the time.

So there are some things that I've formed opinions about that would be harder, I think, to disavow me of.

1 is the superintelligence argument and the existential threat of AI is 1 where I feel pretty confident in my feeling about that 1.

Like I'm willing to hear other arguments, but like I am not particularly moved by the idea that if we're not careful, we will accidentally create a super intelligence that will destroy human life.

Let's get you in trouble and record your own video.

It's like Bill Gates, I think he said like some quote about the internet, that that's just gonna be a small thing.

And I think Steve Ballmer said, I don't know why I'm sticking on Microsoft.

That's something that like smartphones are useless.

There's no reason why Microsoft should get into smartphones, that kind of.

As AGI is destroying the world, we'll look back at this video and see.

Now, I think it's really interesting to actually talk about because nobody really knows the future, so you have to use your best intuition.

It's very difficult to predict it, but you have spoken about AGI and the existential risks around it and sort of based on your intuition that we're quite far away from that being a serious concern relative to the other concerns we have.

So As I understand it, for example, I read Bostrom's book and a bunch of other reading material about this sort of general way of thinking about the world.

And I think the story goes something like this, that we will at some point create computers that are smart enough that they can help design the next version of themselves, which itself will be smarter than the previous version of themselves and eventually bootstrapped up to being smarter than us, at which point we are essentially at the mercy of this sort of more powerful intellect, which in principle, we don't have any control over what its goals are.

And so if its goals are at all out of sync with our goals, like the, for example, the continued existence of humanity, we won't be able to stop it.

It'll be way more powerful than us, and we will be toast.

So there's some, I don't know, very smart people who have signed on to that story.

I once, now I can really get myself in trouble.

I once wrote an op-ed about this, specifically responding to some quotes from Elon Musk, who has been on this very podcast more than once.

there's a- That's a thing he said, but then he came to Providence, Rhode Island, which is where I live, and said to the governors of all the states, you know, you're worried about entirely the wrong thing, you need to be worried about AI.

You need to be very, very worried about AI.

So, and journalists kind of reacted to that and they wanted to get people's take.

And I was like, okay, my belief is that 1 of the things that makes Elon Musk so successful and so remarkable as an individual is that he believes in the power of ideas.

He believes that you can have, you can, if you know, if you have a really good idea for getting into space, you can get into space.

If you have a really good idea for a company or for how to change the way that people drive.

You just have to do it and it can happen.

It's really natural to apply that same idea to AI.

You see these systems that are doing some pretty remarkable computational tricks, demonstrations, and then to take that idea and just push it all the way to the limit and think, okay, where does this go?

And if you're a deep believer in the power of ideas, then it's really natural to believe that those ideas could be taken to the extreme and kill us.

So I think his strength is also his undoing because that doesn't mean it's true.

Like it doesn't mean that that has to happen but it's natural for him to think that.

So another way to phrase the way he thinks and I find it very difficult to argue with that line of thinking.

So Sam Harris is another person from a neuroscience perspective that thinks like that, is saying, well, is there something fundamental in the physics of the universe that prevents this from eventually happening.

And Nick Bostrom thinks in the same way, that kind of zooming out, yeah, okay, we humans now are existing in this time scale of minutes and days, and so our intuition is in this time scale of minutes, hours, and days, but if you look at the span of human history, is there any reason you can't see this in 100 years?

See all Lex Fridman transcripts on Youtube

Michael Littman: Reinforcement Learning and the Future of AI | Lex Fridman Podcast #144