This is a clip from a conversation with Peter Norvig from Sep 2019. New full episodes every Mon & Thu and 1-2 new clips or a new non-podcast video on all other days. You can watch the full conversation here: https://www.youtube.com/watch?v=_VPxEcT_Adc
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on iTunes:
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a few clips every Tue & Fri. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Peter Norvig is a research director at Google and the co-author with Stuart Russell of the book Artificial Intelligence: A Modern Approach that educated and inspired a whole generation of researchers including myself to get into the field.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

Anytime you use neural networks, anytime you learn from data, form representation from data in an automated way, it's not very explainable as to, or it's not introspective to us humans in terms of how this neural network sees the world.

Where, why does it succeed so brilliantly in so many cases and fail so miserably in surprising ways and small.

So what do you think is the future there?

Can simply more data, better data, more organized data solve that problem?

Or is there elements of symbolic systems that need to be brought in, which are a little bit more explainable.

I prefer to talk about trust, and validation, and verification, rather than just about explainability.

Then I think Explanations are 1 tool that you use towards those goals.

And I think it is an important issue that we don't want to use these systems unless we trust them and we want to understand where they work and where they don't work.

And an explanation can be part of that, right?

So I apply for a loan and I get denied, I want some explanation of why.

And you have, in Europe we have the GDPR that says you're required to be able to get that.

But on the other hand, an explanation alone is not enough.

Right, so, you know, we are used to dealing with people and with organizations and corporations and so on, and they can give you an explanation, then you have no guarantee that that explanation relates to reality.

So the bank can tell me, well, you didn't get the loan because you didn't have enough collateral, and that may be true, or it may be true that they just didn't like my religion or something else.

I can't tell from the explanation, And that's true whether the decision was made by a computer or by a person.

I do want to have the explanations and I want to be able to have a conversation to go back and forth and said, well, you gave this explanation, but what about this?

And what would have happened if this had happened?

So I think a conversation is a better way to think about it than just an explanation as a single output.

And I think we need testing of various kinds.

In order to know, was the decision really based on my collateral or was it based on my religion or skin color or whatever?

I can't tell if I'm only looking at my case, but if I look across all the cases, then I can detect a pattern.

So you want to have that kind of capability.

You want to have these adversarial testing.

So we thought we were doing pretty good at object recognition in images.

We said, look, we're at sort of pretty close to human level performance on ImageNet and so on.

And then you start seeing these adversarial images and you say, wait a minute, that part is nothing like human performance.

You can mess with it really easily, right?

And yeah, you can do that to humans too, right?

Right, humans don't know what color the dress was.

And so they're vulnerable to certain attacks that are different than the attacks on the machines.

But the attacks on the machines are so striking.

They really change the way you think about what we've done.

The way I think about it is, I think part of the problem is we're seduced by our low dimensional metaphors.

You look in a textbook and you say, okay, now we've mapped out the space and a cat is here and dog is here and maybe there's a tiny little spot in the middle where you can't tell the difference, but mostly we've got it all covered.

And if you believe that metaphor, then you say, well, we're nearly there.

And there's only gonna be a couple adversarial images.

And what you should really say is, it's not a 2D flat space that we've got mostly covered.

And a cat is this string that goes out in this crazy path.

And if you step a little bit off the path in any direction, you're in nowhere's land and you don't know what's going to happen.

I think that's where we are and now we've got to deal with that.

It wasn't so much an explanation, but it was an understanding of what the models are and what they're doing, and now we can start exploring how do you fix that.

Yeah, validating the robustness of the system, so on.

Do you think we're a little too hard on our robots in terms of the standards we apply?

So, you know, there's a dance, there's a dance in nonverbal and verbal communication between humans.

You know, if we apply the same kind of standard in terms of humans, you know, we trust each other pretty quickly.

You know, you and I haven't met before and there's some degree of trust, right, that nothing's going to go crazy wrong.

And yet to AI, when we look at AI systems, we seem to approach through skepticism always, always.

And it's like they have to prove through a lot of hard work that they're even worthy of even inkling of our trust.

How do we break that barrier, close that gap?

Just listening, my friend Mark Moffat is a naturalist and he says the most amazing thing about humans is that you can walk into a coffee shop or a busy street in a city, and there's lots of people around you that you've never met before, and you don't kill each other.

If a chimpanzee's in a situation where here's some that aren't from my tribe, bad things happen.

Especially in a coffee shop, there's delicious food around, you know.

Yeah, but we humans have figured that out, right?

We still go to war, we still do terrible things, but for the most part, we've learned to trust each other and live together.

So that's gonna be important for our AI systems as well.

And also I think, you know, a lot of the emphasis is on AI, but in many cases, AI is part of the technology, but isn't really the main thing.

So a lot of what we've seen is more due to communications technology than AI technology.

Yeah, you want to make these good decisions, but the reason we're able to have any kind of system at all is we've got the communications so that we're collecting the data and so that we can reach lots of people around the world.

I think that's a bigger change that we're dealing with.


Speaker 0: Anytime you use neural networks, anytime you learn from data, form representation from data in an automated way, it's not very explainable as to, or it's not introspective to us humans in terms of how this neural network sees the world. Where, why does it succeed so brilliantly in so many cases and fail so miserably in surprising ways and small. So what do you think is the future there? Can simply more data, better data, more organized data solve that problem? Or is there elements of symbolic systems that need to be brought in, which are a little bit more explainable.

Speaker 1: Yeah. I prefer to talk about trust, and validation, and verification, rather than just about explainability. Then I think Explanations are 1 tool that you use towards those goals. And I think it is an important issue that we don't want to use these systems unless we trust them and we want to understand where they work and where they don't work. And an explanation can be part of that, right?

So I apply for a loan and I get denied, I want some explanation of why. And you have, in Europe we have the GDPR that says you're required to be able to get that. But on the other hand, an explanation alone is not enough. Right, so, you know, we are used to dealing with people and with organizations and corporations and so on, and they can give you an explanation, then you have no guarantee that that explanation relates to reality. Right?

Right? So the bank can tell me, well, you didn't get the loan because you didn't have enough collateral, and that may be true, or it may be true that they just didn't like my religion or something else. I can't tell from the explanation, And that's true whether the decision was made by a computer or by a person. So I want more. I do want to have the explanations and I want to be able to have a conversation to go back and forth and said, well, you gave this explanation, but what about this?

And what would have happened if this had happened? And what would I need to change that? So I think a conversation is a better way to think about it than just an explanation as a single output. And I think we need testing of various kinds. In order to know, was the decision really based on my collateral or was it based on my religion or skin color or whatever?

I can't tell if I'm only looking at my case, but if I look across all the cases, then I can detect a pattern. So you want to have that kind of capability. You want to have these adversarial testing. So we thought we were doing pretty good at object recognition in images. We said, look, we're at sort of pretty close to human level performance on ImageNet and so on.

And then you start seeing these adversarial images and you say, wait a minute, that part is nothing like human performance.

Speaker 0: And you can mess with it really easily.

Speaker 1: You can mess with it really easily, right? And yeah, you can do that to humans too, right?

Speaker 0: In a different way, perhaps.

Speaker 1: Right, humans don't know what color the dress was. Right. And so they're vulnerable to certain attacks that are different than the attacks on the machines. But the attacks on the machines are so striking. They really change the way you think about what we've done.

The way I think about it is, I think part of the problem is we're seduced by our low dimensional metaphors.

Speaker 0: Yeah. I like that phrase.

Speaker 1: You look in a textbook and you say, okay, now we've mapped out the space and a cat is here and dog is here and maybe there's a tiny little spot in the middle where you can't tell the difference, but mostly we've got it all covered. And if you believe that metaphor, then you say, well, we're nearly there. And there's only gonna be a couple adversarial images. But I think that's the wrong metaphor. And what you should really say is, it's not a 2D flat space that we've got mostly covered.

It's a million dimension space. And a cat is this string that goes out in this crazy path. And if you step a little bit off the path in any direction, you're in nowhere's land and you don't know what's going to happen. I think that's where we are and now we've got to deal with that. It wasn't so much an explanation, but it was an understanding of what the models are and what they're doing, and now we can start exploring how do you fix that.

Speaker 0: Yeah, validating the robustness of the system, so on. But take it back to this word trust. Do you think we're a little too hard on our robots in terms of the standards we apply? So, you know, there's a dance, there's a dance in nonverbal and verbal communication between humans. You know, if we apply the same kind of standard in terms of humans, you know, we trust each other pretty quickly.

You know, you and I haven't met before and there's some degree of trust, right, that nothing's going to go crazy wrong. And yet to AI, when we look at AI systems, we seem to approach through skepticism always, always. And it's like they have to prove through a lot of hard work that they're even worthy of even inkling of our trust. What do you think about that? How do we break that barrier, close that gap?

Speaker 1: I think that's right. I think that's a big issue. Just listening, my friend Mark Moffat is a naturalist and he says the most amazing thing about humans is that you can walk into a coffee shop or a busy street in a city, and there's lots of people around you that you've never met before, and you don't kill each other.

Speaker 0: Yeah.

Speaker 1: He says, chimpanzees cannot do that. Yeah, right. Right. If a chimpanzee's in a situation where here's some that aren't from my tribe, bad things happen.

Speaker 0: Especially in a coffee shop, there's delicious food around, you know.

Speaker 1: Yeah, but we humans have figured that out, right? And you know, for the most part. We still go to war, we still do terrible things, but for the most part, we've learned to trust each other and live together. So that's gonna be important for our AI systems as well. And also I think, you know, a lot of the emphasis is on AI, but in many cases, AI is part of the technology, but isn't really the main thing.

So a lot of what we've seen is more due to communications technology than AI technology. Yeah, you want to make these good decisions, but the reason we're able to have any kind of system at all is we've got the communications so that we're collecting the data and so that we can reach lots of people around the world. I think that's a bigger change that we're dealing with. You

See all Lex Fridman transcripts on Youtube

Peter Norvig: We Are Seduced by Our Low-Dimensional Metaphors | AI Podcast Clips