The following is a conversation with Peter Norvig.

He's the director of research at Google and the co-author with Stuart Russell of the book Artificial Intelligence, A Modern Approach that educated and inspired a whole generation of researchers, including myself, to get into the field of artificial intelligence.

This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube, give it 5 stars on iTunes, support on Patreon, or simply connect with me on Twitter, at Lex Friedman, spelled F-R-I-D-M-A-N.

And now, here's my conversation with Peter Norvig.

Most researchers in the AI community, including myself, own all 3 editions, red, green, and blue, of the Artificial Intelligence, A Modern Approach.

It's a field-defining textbook, as many people are aware, that you wrote with Stuart Russell.

How has the book changed, and how have you changed, in relation to it, from the first edition, to the second, to the third, and now fourth edition, as you work on it?

Yeah, So it's been a lot of years, a lot of changes.

1 of the things changing from the first to maybe the second or third was just the rise of computing power.

So I think in the in the first edition, we said, here's predicate logic.

But that only goes so far, because pretty soon, you have millions of short little predicate expressions, and they can possibly fit in memory.

So we're going to use first-order logic that's more concise.

And then we quickly realized, oh, predicate logic is pretty nice because there are really fast SAT solvers and other things.

And look, there's only millions of expressions and that fits easily into memory, or maybe even billions fit into memory now.

That was a change of the type of technology we needed just because the hardware expanded.

Resource constraints were loosened significantly for the second.

I think we're starting to see that again with the GPUs and then more specific type of machinery like the TPUs and using custom ASICs and so on for deep learning.

So we're seeing another advance in terms of the hardware.

Then I think another thing that we especially notice this time around is in all 3 of the first editions, we said, well, we're going to find AI as maximizing expected utility, and you tell me your utility function, and now we've got 27 chapters worth of cool techniques for how to optimize that.

I think in this edition, we're saying more, you know what, maybe that optimization part is the easy part, and the hard part is deciding what is my utility function?

And if I'm a collection of agents or a society, what do we want as a whole?

So you touched that topic in this edition.

On a technical level, we're almost pushing the philosophical.

So we've always had a philosophy chapter, which I was glad that we were supporting.

And now it's less kind of the, you know, Chinese room type argument, and more of these ethical and societal type issues.

So we get into the issues of fairness and bias, and just the issue of aggregating utilities.

So how do you encode human values into a utility function?

Is this something that you can do purely through data in a learned way, or is there some systematic?

There's just beginnings to even opening the doors.

Yes, there are techniques to try to learn that.

So we talk about inverse reinforcement learning, right?

So reinforcement learning, you take some actions, you get some rewards, and you figure out what actions you should take.

In inverse reinforcement learning, you observe somebody taking actions and you figure out, well, this must be what they were trying to do.

If they did this action, it must be because they wanted.

So lots of people take actions that are self-destructive or they're suboptimal in certain ways.

You want to somehow learn the perfect actions rather than the ones they actually take.

Then another big part of it is just kind of theoretical of saying what can we accomplish.

And so you look at like this work on the programs to predict recidivism and decide who should get parole or who should get bail or whatever.

And 1 of the big issues is fairness across protected classes, protected classes being things like sex and race and so on.

2 things you want is you want to say, well, if I get a score of say 6 out of

then I want that to mean the same, whether no matter what race I'm on.

So I want to have a 60 percent chance of reoccurring regardless.

1 of the makers of a commercial program to do that says that's what we're trying to optimize and look, we achieved that.

And then on the other side, you also want to say, well, If it makes mistakes, I want that to affect both sides of the protected class equally.

And it turns out they don't do that, right?

So they're twice as likely to make a mistake that would harm a black person over a white person.

So you'd like to say, well I want to achieve both those goals.

And then it turns out you do the analysis and it's theoretically impossible to achieve both those goals.

So you have to trade them off, 1 against the other.

So that analysis is really helpful to know what you can aim for and how much you can get.

But the analysis certainly can't tell you where should we make that trade-off point.

But nevertheless, then we can, as humans, deliberate where that trade-off should be.

Yeah, so at least we now, we're arguing in an informed way.

We're not asking for something impossible.

We're saying, here's where we are and here's what we aim for and this strategy is better than that strategy.

So that's, I would argue, is a really powerful and really important first step, but it's a doable 1, sort of removing undesirable degrees of bias in systems in terms of protected classes.

And then there's something, I listened to your commencement speech, there's some fuzzier things like You mentioned angry birds.

Do you want to create systems that feed the dopamine enjoyment, that feed, that optimize for you returning to the system, enjoying the moment of playing the game, of getting likes or whatever, this kind of thing, or some kind of long-term improvement.

That's really going to the philosophical area.

I think that's a really important issue too, certainly thinking about that.

I don't think about that as an AI issue as much.

But as you say, the point is we've built this society and this infrastructure where we say we have a marketplace for attention and we've decided as a society that we like things that are free and so we want all apps on our phone to be free and that means they're all competing for your attention and then eventually they make some money some way through ads or in-game sales or whatever.

But they can only win by defeating all the other apps by instilling your attention.

And we built a marketplace where it seems like they're working against you rather than working with you.

And I'd like to find a way where we can change the playing field so you feel more like, well, these things are on my side.

Yes, they're letting me have some fun in the short term, but they're also helping me in the long term, rather than competing against me.

And those aren't necessarily conflicting objectives, they're just the incentives, the direct current incentives as we try to figure out this whole new world, it seemed to be on the easier part of that, which is feeding the dopamine the rush.

But, so maybe take a quick step back at the beginning of the Artificial Intelligence and Modern Approach book of writing.

So here you are in the 90s, when you first sat down with Stuart to write the book to cover an entire field, which is 1 of the only books that has successfully done that for AI and actually in a lot of other computer science fields, it's a huge undertaking.

Did you envision that you would be trying to cover the entire field?

Was there a systematic approach to it that was more step by step?

So I guess it came about, you know, I'd go to lunch with the other AI faculty at Berkeley and we'd say, you know, the field is changing, seems like the current books are a little bit behind.

Nobody's come out with a new book recently.

And everybody said, yeah, yeah, that's a great thing to do.

And then I ended up heading off to industry.

I went to Sun Lab, so I thought, well, that's the end of my possible academic publishing career.

But I met Stuart again at a conference like a year later and said, you know that book we were always talking about?

You guys must be half done with it by now, right?

And he said, well, we keep talking, we never do anything.

So I said, well, you know, we should do it.

And I think the reason is that we all felt it was a time where the field was changing.

So the good old fashioned AI was based primarily on Boolean logic, and you had a few tricks to deal with uncertainty.

And it was based primarily on knowledge engineering, that the way you got something done is you went out, you interviewed an expert, and you wrote down by hand everything they knew.

And we saw in 95 that the field was changing in 2 ways.

1, we were moving more towards probability rather than Boolean logic, and we were moving more towards machine learning rather than knowledge engineering.

And the other books hadn't caught that wave.

They were still more in the old school, so certainly they had part of that on the way.

But we said, if we start now completely taking that point of view, we can have a different kind of book, and we were able to put that together.

And what was literally the process, if you remember?

Did you start writing a chapter, Did you outline?

Yeah, I guess we did an outline and then we assigned chapters to each person.

At the time, I had moved to Boston and Stuart was in Berkeley.

Basically, we did it over the Internet, and that wasn't the same as doing it today.

It meant dial-up lines and telnetting in.

You telnet it into 1 shell and you type cat file name and you hoped it was captured at the other end.

And certainly you're not sending images and figures back and forth.

But Did you anticipate where the field would go from that day, from the 90s?

Did you see the growth into learning-based methods, into data-driven methods that followed in the future decades?

We certainly thought that learning was important.

I guess we missed it as being as important as it is today.

We missed that The idea of deep learning hadn't been invented yet.

We could have taken the book from a complete machine learning point of view right from the start.

We chose to do it more from a point of view of we're going to first develop different types of representations and we're gonna talk about different types of environments.

Is it fully observable or partially observable and is it deterministic or stochastic and so on and we made it more complex along those axes, rather than focusing on the machine learning axis first.

Do you think, you know, there's some sense in which the deep learning craze is extremely successful for a particular set of problems.

And, you know, eventually it's going to, in the general case, hit challenges.

So in terms of the difference between perception systems and robots that have to act in the world, do you think we're going to return to AI, a modern approach type breadth in addition 5 and 6 in future decades?

Do you think deep learning will take its place as a chapter in this bigger view of AI?

Yeah, I think we don't know yet how it's all going to play out.

So in the new edition, we have a chapter on deep learning.

We got Ian Goodfellow to be the guest author for that chapter.

So he said he could condense his whole deep learning book into 1 chapter.

I think he did a great job We were also encouraged that he you know, we gave him the old neural net chapter and said

modernize that and he said, you know half of that was okay that Certainly, there's lots of new things that have been developed, but some of the core was still the same.

I think we'll gain a better understanding of what you can do there.

I think we'll need to incorporate all the things we can do with the other technologies.

So deep learning started out, convolutional networks and very close to perception.

It's since moved to be able to do more with actions and some degree of longer term planning.

But we need to do a better job with representation and reasoning and 1 shot learning and so on.

And I think we don't know yet how that's going to play out.

So do you think looking at the, some success, but certainly eventual demise, a partial demise of experts of symbolic systems in the 80s.

Do you think there is kernels of wisdom in the work that was done there with logic and reasoning and so on that will rise again in your view?

So certainly I think the idea of representation and reasoning is crucial that sometimes you just don't have enough data about the world to learn de novo.

So you've got to have some idea of representation, whether that was programmed in or told or whatever, and then be able to take steps of reasoning.

I think the problem with the good old-fashioned AI was, 1, we tried to base everything on these symbols that were atomic, and that's great if you're like trying to define the properties of a triangle, right?

Because they have necessary and sufficient conditions.

The real world is messy and doesn't have sharp edges, and atomic symbols do.

And then the other aspect was that the reasoning was universal and applied anywhere, which in some sense is good, but it also means there's no guidance as to where to apply.

And so you started getting these paradoxes like, well, if I have a mountain and I remove 1 grain of sand, then it's still a mountain.

But if I do that repeatedly, at some point it's not.

And with logic, there's nothing to stop you from applying things repeatedly.

But maybe with something like deep learning, and I don't really know what the right name for it is, we could separate out those ideas.

So 1, we could say, a mountain isn't just an atomic notion, it's something like a word embedding that has a more complex representation.

Secondly, we could somehow learn, yeah, there's this rule that you can remove 1 grain of sand, and you can do that a bunch of times, but you can't do it a near infinite amount of times.

But on the other hand, when you're doing induction on the integers, sure, then it's fine to do it an infinite number of times.

And if we could, somehow we have to learn when these strategies are applicable, rather than having the strategies be completely neutral and available everywhere.

Anytime you use neural networks, anytime you learn from data, form representation from data in an automated way, it's not very explainable as to, or it's not introspective to us humans in terms of how this neural network sees the world, where why does it succeed so brilliantly in so many cases and fail so miserably in surprising ways and small.

So what do you think is the future there?

Can simply more data, better data, more organized data solve that problem?

Or is there elements of symbolic systems that need to be brought in, which are a little bit more explainable?

So I prefer to talk about trust, and validation, and verification, rather than just about explainability.

Then I think explanations are 1 tool that you use towards those goals.

I think it is an important issue that we don't want to use these systems unless we trust them and we want to understand where they work and where they don't work.

So, I apply for a loan and I get denied, I want some explanation of why.

And you have, in Europe, we have the GDPR that says you're required to be able to get that.

But on the other hand an explanation alone is not enough right so you know we were used to dealing with people and with organizations and corporations and so on, and they can give you an explanation, then you have no guarantee that that explanation relates to reality.

So the bank can tell me, well, you didn't get the loan because you didn't have enough collateral, and that may be true, or it may be true that they just didn't like my religion or or something else I can't tell from the explanation and that's that's true whether the decision was made by a computer or by a person so I want more I do want to have the explanations and I want to be able to have a conversation to go back and forth and said, well, you gave this explanation, but what about this?

What would have happened if this had happened?

So I think a conversation is a better way to think about it than just an explanation as a single output.

I think we need testing of various kinds.

So in order to know, Was the decision really based on my collateral or was it based on my religion or skin color or whatever?

I can't tell if I'm only looking at my case, but if I look across all the cases, then I can detect a pattern.

So you want to have that kind of capability.

You want to have these adversarial testing.

So we thought we were doing pretty good at object recognition in images.

We said, look, we're at pretty close to human level performance on ImageNet and so on.

Then you start seeing these adversarial images and you say, wait a minute, that part is nothing like human performance.

You can mess with it really easily, right?

And yeah, you can do that to humans too, right?

Right, humans don't know what color the dress was.

And so they're vulnerable to certain attacks that are different than the attacks on the machines.

But the attacks on the machines are so striking, they really change the way you think about what we've done, right?

And the way I think about it is I think part of the problem is we're seduced by our low dimensional metaphors, right?

You look in a textbook and you say, okay, now we've mapped out the space.

And you know, a cat is here and dog is here.

And maybe there's a tiny little spot in the middle where you can't tell the difference, but mostly we've got it all covered.

And if you believe that metaphor, then you say, well, we're nearly there.

And there's only gonna be a couple adversarial images.

And what you should really say is, it's not a 2D flat space that we've got mostly covered.

And a cat is this string that goes out in this crazy path.

And if you step a little bit off the path in any direction, you're in nowhere's land and you don't know what's going to happen.

So it wasn't so much an explanation, but it was an understanding of what the models are and what they're doing and now we can start exploring how do you fix that.

Yeah, validating the robustness of the system and so on.

Do you think we're a little too hard on our robots in terms of the standards we apply?

So, you know, of, there's a dance, There's a dance in nonverbal and verbal communication between humans.

You know, if we apply the same kind of standard in terms of humans, you know, we trust each other pretty quickly.

You know, you and I haven't met before and there's some degree of trust, that nothing's gonna go crazy wrong.

And yet to AI, when we look at AI systems, we seem to approach through skepticism always, always.

And it's like they have to prove through a lot of hard work that they're even worthy of even inkling of our trust.

How do we break that barrier, close that gap?

Just listening, my friend Mark Moffitt is a naturalist and he says the most amazing thing about humans is that you can walk into a coffee shop or a busy street in a city and there's lots of people around you that you've never met before and you don't kill each other.

Right, if a chimpanzee's in a situation where here's some that aren't from my tribe, Bad things happen.

Especially in a coffee shop, there's delicious food around.

We still go to war, we still do terrible things, but for the most part, we've learned to trust each other and live together.

So that's gonna be important for our AI systems as well.

And also I think a lot of the emphasis is on AI, but in many cases AI is part of the technology, but isn't really the main thing.

So a lot of what we've seen is more due to communications technology than AI technology.

Yeah, you want to make these good decisions, But the reason we're able to have any kind of system at all is we've got the communication so that we're collecting the data and so that we can reach lots of people around the world.

I think that's a bigger change that we're dealing with.

Speaking of reaching a lot of people around the world, on the side of education, 1 of the many things in terms of education you've done, you've taught the Intro to Artificial Intelligence course that signed up 100,000, 160,000 students.

It was 1 of the first successful examples of a MOOC, Massive Open Online Course.

What do you think is the future of MOOCs, of education online?

Yeah, it was a great fun doing it, particularly being right at the start, just because it was exciting and new, but it also meant that we had less competition.

1 of the things you hear about, well, the problem with MOOCs is the completion rates are so low, so there must be a failure.

I probably started 50 different courses that I haven't finished, but I got exactly what I wanted out of them, because I had never intended to finish them.

I just wanted to dabble in a little bit either to see the topic matter or just to see the pedagogy of how are they doing this class.

So I guess the main thing I learned is when I came in I thought the challenge was information.

Saying if I'm just take the stuff I want you to know, and I'm very clear and explain it well, then my job is done and good things are gonna happen.

And then in doing the course, I learned, well, yeah, you got to have the information, but really the motivation is the most important thing.

That if students don't stick with it, it doesn't matter how good the content is.

See all Lex Fridman transcripts on Youtube

Peter Norvig: Artificial Intelligence: A Modern Approach | Lex Fridman Podcast #42