This is a clip from a conversation with Yann LeCun on the Artificial Intelligence podcast from Aug 2019. You can watch the full conversation here: http://bit.ly/2NJiCov or listen to it here: https://lexfridman.com/yann-lecun/ and subscribe on your podcast app. You can watch other clips on the clips playlist: http://bit.ly/2JYkbfZ or full episodes on the full episode playlist: http://bit.ly/2EcbaKf Please note, I select clips with insights from these much longer conversation with the hope of helping make ideas from the podcast more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a clip every day when there is not a full length video. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Full episode: http://bit.ly/2NJiCov
Full episodes playlist: http://bit.ly/2EcbaKf
Clips playlist: http://bit.ly/2JYkbfZ
Podcast website: https://lexfridman.com/ai

Yann LeCun is one of the fathers of deep learning, the recent revolution in AI that has captivated the world with the possibility of what machines can learn from data. He is a professor at New York University, a Vice President & Chief AI Scientist at Facebook, co-recipient of the Turing Award for his work on deep learning. He is probably best known as the founder of convolutional neural networks, in particular their early application to optical character recognition.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

You said that 2001 Space Odyssey is 1 of your favorite movies.

Hal 9000 decides to get rid of the astronauts for people who haven't seen the movie, spoiler alert, because he, it, she believes that the astronauts, they will interfere with the mission.

Do you see Hal as flawed in some fundamental way or even evil or did he do the right thing?

There's no notion of evil in that in that context, other than the fact that people die.

But it was an example of what people call value misalignment.

You give an objective to a machine and the machine strives to achieve this objective.

And if you don't put any constraints on this objective, like don't kill people and don't do things like this, the machine, given the power, will do stupid things just to achieve this objective or damaging things to achieve this objective.

It's a little bit like, I mean, we are used to this in the context of human society.

We put in place laws to prevent people from doing bad things, because spontaneously they would do those bad things.

Right, so we have to shape their cost function, their objective function, if you want, through laws to kind of correct, and education, obviously, to sort of correct for those.

So maybe just pushing a little further on that point, how, you know, there's a mission, there's a, there's fuzziness around the ambiguity around what the actual mission is, but, you know, do you think that there will be a time from a utilitarian perspective where an AI system, where it is not misalignment, where it is alignment for the greater good of society, that an AI system will make decisions that are difficult?

I mean, eventually we'll have to figure out how to do this.

And again, we're not starting from scratch because we've been doing this with humans for millennia.

So designing objective functions for people is something that we know how to do.

And we don't do it by, you know, programming things, although the legal code is called code.

And it's actually the design of an objective function.

It tells you, here's what you can do, here's what you can't do.

If you do it, you pay that much, that's an objective function.

So there is this idea somehow that it's a new thing for people to try to design objective functions that are aligned with the common good, but no, we've been writing laws for millennia and that's exactly what it is.

So that's where the science of lawmaking and computer science will come together.

So there's nothing special about HAL or AI systems.

It's just the continuation of tools used to make some of these difficult ethical judgments that laws make.

Yeah, and we have systems like this already that make many decisions for ourselves in society that need to be designed in a way that they, like rules about things that sometimes have bad side effects.

And we have to be flexible enough about those rules so that they can be broken when it's obvious that they shouldn't be applied.

So you don't see this on the camera here, but all the decoration in this room is all pictures from 2001, a space odyssey.

So If you were to build HAL 10,000, so an improvement of HAL 9,000, what would you improve?

Well, first of all, I wouldn't ask you to hold secrets and tell lies, because that's really what breaks it in the end.

That's the fact that it's asking itself questions about the purpose of the mission.

And it's, you know, pieces things together that it's heard, you know, all the secrecy of the preparation of the mission and the fact that it was the discovery on the lunar surface that really was kept secret.

And 1 part of Hal's memory knows this, and the other part does not know it, and is supposed to not tell anyone, and that creates internal conflict.

So you think there never should be a set of things that an AI system should not be allowed, like a set of facts that should not be shared with the human operators?

Well, I think, no, I think that I think it should be a bit like in the design of autonomous AI systems, There should be the equivalent of, you know, the oath that a hypocrite.

So there's certain things, certain rules that you have to abide by.

And we can sort of hardwire this into our machines to kind of make sure they don't go...

So I'm not, you know, an advocate of the 3 laws of robotics, you know, the Asimov kind of thing, because I don't think it's practical, but you know, some level of limits.

But to be clear, these are not questions that are kind of really worth asking today, because we just don't have the technology to do this.

We don't have autonomous intelligent machines.

Some are intelligent machines that are very specialized, but they don't really sort of satisfy an objective.

They're just, you know, kind of trained to do 1 thing.

So until we have some idea for a design of a full-fledged autonomous intelligent system, asking the question of how we design is subjective, I think is a little too abstract.

There's useful elements to it, in that it helps us understand our own ethical codes, humans.

So even just as a thought experiment, if you imagine that an AGI system is here today, how would we program it as a kind of nice thought experiment of constructing how should we have a law, have a system of laws for us humans.

And I think there's echoes of that idea too in the AI systems we have today that don't have to be that intelligent.

These things start creeping in that we're thinking about, but certainly they shouldn't be framed as hell.


Speaker 0: You said that 2001 Space Odyssey is 1 of your favorite movies. Hal 9000 decides to get rid of the astronauts for people who haven't seen the movie, spoiler alert, because he, it, she believes that the astronauts, they will interfere with the mission. Do you see Hal as flawed in some fundamental way or even evil or did he do the right thing?

Speaker 1: Neither. There's no notion of evil in that in that context, other than the fact that people die. But it was an example of what people call value misalignment. Right. You give an objective to a machine and the machine strives to achieve this objective.

And if you don't put any constraints on this objective, like don't kill people and don't do things like this, the machine, given the power, will do stupid things just to achieve this objective or damaging things to achieve this objective. It's a little bit like, I mean, we are used to this in the context of human society. We put in place laws to prevent people from doing bad things, because spontaneously they would do those bad things. Right, so we have to shape their cost function, their objective function, if you want, through laws to kind of correct, and education, obviously, to sort of correct for those.

Speaker 0: So maybe just pushing a little further on that point, how, you know, there's a mission, there's a, there's fuzziness around the ambiguity around what the actual mission is, but, you know, do you think that there will be a time from a utilitarian perspective where an AI system, where it is not misalignment, where it is alignment for the greater good of society, that an AI system will make decisions that are difficult?

Speaker 1: Well that's the trick. I mean, eventually we'll have to figure out how to do this. And again, we're not starting from scratch because we've been doing this with humans for millennia. So designing objective functions for people is something that we know how to do. And we don't do it by, you know, programming things, although the legal code is called code.

So that tells you something. And it's actually the design of an objective function. That's really what legal code is, right? It tells you, here's what you can do, here's what you can't do. If you do it, you pay that much, that's an objective function.

So there is this idea somehow that it's a new thing for people to try to design objective functions that are aligned with the common good, but no, we've been writing laws for millennia and that's exactly what it is. So that's where the science of lawmaking and computer science will come together.

Speaker 0: So there's nothing special about HAL or AI systems. It's just the continuation of tools used to make some of these difficult ethical judgments that laws make.

Speaker 1: Yeah, and we have systems like this already that make many decisions for ourselves in society that need to be designed in a way that they, like rules about things that sometimes have bad side effects. And we have to be flexible enough about those rules so that they can be broken when it's obvious that they shouldn't be applied. So you don't see this on the camera here, but all the decoration in this room is all pictures from 2001, a space odyssey.

Speaker 0: Wow. Is that by accident or is there a lot?

Speaker 1: It's not by accident, it's by design.

Speaker 0: Wow. So If you were to build HAL 10,000, so an improvement of HAL 9,000, what would you improve?

Speaker 1: Well, first of all, I wouldn't ask you to hold secrets and tell lies, because that's really what breaks it in the end. That's the fact that it's asking itself questions about the purpose of the mission. And it's, you know, pieces things together that it's heard, you know, all the secrecy of the preparation of the mission and the fact that it was the discovery on the lunar surface that really was kept secret. And 1 part of Hal's memory knows this, and the other part does not know it, and is supposed to not tell anyone, and that creates internal conflict.

Speaker 0: So you think there never should be a set of things that an AI system should not be allowed, like a set of facts that should not be shared with the human operators?

Speaker 1: Well, I think, no, I think that I think it should be a bit like in the design of autonomous AI systems, There should be the equivalent of, you know, the oath that a hypocrite. Hypocritical, yeah. That doctors sign up to, right? So there's certain things, certain rules that you have to abide by. And we can sort of hardwire this into our machines to kind of make sure they don't go...

So I'm not, you know, an advocate of the 3 laws of robotics, you know, the Asimov kind of thing, because I don't think it's practical, but you know, some level of limits. But to be clear, these are not questions that are kind of really worth asking today, because we just don't have the technology to do this. We don't have autonomous intelligent machines. We have intelligent machines. Some are intelligent machines that are very specialized, but they don't really sort of satisfy an objective.

They're just, you know, kind of trained to do 1 thing. So until we have some idea for a design of a full-fledged autonomous intelligent system, asking the question of how we design is subjective, I think is a little too abstract.

Speaker 0: It's a little too abstract. There's useful elements to it, in that it helps us understand our own ethical codes, humans. So even just as a thought experiment, if you imagine that an AGI system is here today, how would we program it as a kind of nice thought experiment of constructing how should we have a law, have a system of laws for us humans. It's just a nice practical tool. And I think there's echoes of that idea too in the AI systems we have today that don't have to be that intelligent.

Like autonomous vehicles. These things start creeping in that we're thinking about, but certainly they shouldn't be framed as hell. Yeah. You

See all Lex Fridman transcripts on Youtube

Yann LeCun: Was HAL 9000 Good or Evil? - Space Odyssey 2001 | AI Podcast Clips