Full episode with Ben Goertzel (Jun 2020): https://www.youtube.com/watch?v=OpSmCKe27WE
Clips channel (Lex Clips): https://www.youtube.com/lexclips
Main channel (Lex Fridman): https://www.youtube.com/lexfridman
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on Apple Podcasts (iTunes):
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Ben Goertzel is one of the most interesting minds in the artificial intelligence community. He is the founder of SingularityNET, designer of OpenCog AI framework, formerly a director of the Machine Intelligence Research Institute, Chief Scientist of Hanson Robotics, the company that created the Sophia Robot. He has been a central figure in the AGI community for many years, including in the Conference on Artificial General Intelligence.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

Maybe it's good to step back a little bit.

People often cite you as the creator, or at least the popularizer of the term AGI, artificial general intelligence.

Can you tell the origin story of the term?

I launched the term AGI upon the world for what it's worth without ever fully being in love with the term.

What happened is I was editing a book, and this process started around 2001 or 2, I think the book came out 2005, if I'm not mistaken.

I was editing a book which I provisionally was titling Real AI.

mean, the goal was to gather together fairly serious academicish papers on the topic of making thinking machines that could really think in the sense like people can or even more broadly than people can, right?

So then I was reaching out to other folks that I had encountered here or there who were interested in that, which included some other folks who I knew from the transhumanist and singularitarian world, like Peter Vos, who has a company, AGI Incorporated, still in California, and included Shane Legg, who had worked for me at my company WebMind in New York in the late 90s, who by now has become rich and famous.

at that time, Shane was, I think he may have just started doing his PhD with Markus Hutter, who at that time hadn't yet published his book, universal AI, which sort of gives a mathematical foundation for artificial general intelligence.

So I reached out to Shane and Marcus and Peter Vos and Pei Wang, who was another former employee of mine who had been Douglas Hofstadter's PhD student, who had his own approach to AGI, and a bunch of some Russian folks reached out to these guys and they contributed papers for the book.

But that was my provisional title, but I never loved it because in the end, I was doing some, what we would now call narrow AI as well, like applying machine learning to genomics data or chat data for sentiment analysis.

Ray Kurzweil wrote about narrow AI versus strong AI.

But that seemed weird to me because, first of all, narrow and strong are not antinodes.

But secondly, strong AI was used in the cognitive science literature to mean the hypothesis that digital computer AIs could have true consciousness like human beings.

So there was already a meaning to strong AI, which was complexly different but related.

So we were tossing around on an email list whether what title it should be, and so we talked about narrow AI, broad AI, wide AI, narrow AI, general AI, and I think it was either Shane Legg or Peter Vos on the private email discussion we had, he said, well, why don't we go with AGI, artificial general intelligence.

Pei Wang wanted to do GAI, General Artificial Intelligence, because in Chinese it goes in that order.

But we figured gay wouldn't work in US culture at that time.

So we went with the AGI, We used it for the title

And part of Peter and Shane's reasoning was you have the G factor in psychology, which is IQ, general intelligence, right?

So you have a meaning of GI, general intelligence in psychology, so then you're looking like artificial GI.

And so I think, maybe both Shane and Peter think they invented the term, but then later after the book was published, this guy Mark Gubrid came up to me and he's like, well, I published an essay with the term AGI in like 1997 or something.

And so I'm just waiting for some Russian to come out

I mean, that term is not dramatically innovative or anything.

It's 1 of these obvious in hindsight things, which is also

annoying in a way, because, you know, Joe Chabac, who you interviewed, is a close friend of mine.

He likes the term synthetic intelligence, which I like much better, but it hasn't actually caught on, right?

a bit off to me because Artifice is like a tool or something, but not all AGI's are gonna be tools.

I mean, they may be now, but we're aiming toward making them agents rather than tools.

And in a way, I don't like the distinction between artificial and natural, because I mean, we're part of nature also, and machines are part of nature.

You can look at evolved versus engineered, but that's a different distinction.

Then it should be engineered general intelligence.

And then general, well, if you look at Marcus Hutter's book, Universally, what he argues there is, within the domain of computation theory, which is limited but interesting, so if you assume computable environments and computable reward functions, then he articulates what would be a truly general intelligence,

a system called AIXI, which is quite beautiful.

AIXI, and that's the middle name of my latest child, actually.

First name is QRXI, Q-R-X-I, which my wife came up with, but that's an acronym for quantum organized rational expanding intelligence.

And his middle name is Exiphanes, actually, which means the former principal underlying AXE.

case- You're giving Elon Musk's new child a run for his money.

He copied me with this new freakish name.

But now if I have another baby, I'm going to have to outdo him.

It's becoming an arms race of weird, geeky baby names.

We'll see what the babies think about it.

But I mean, my oldest son, Zarathustra, loves his name, and my daughter, Sherazade, loves her name.

give your kids weird names, then you're obliged to make the kids weird enough that they like the names, right?

It directs their upbringing in a certain way.

But yeah, anyway, I mean, what Marcus showed in that book is that a truly general intelligence theoretically is possible but would take infinite computing power.

The general is not really achievable within physics as we know it.

I mean, physics as we know it may be limited, but that's what we have to work with now.

Infinitely general, you mean, like information processing perspective, yeah.

Yeah, intelligence is not very well defined either.

I mean, in AI now, it's fashionable to look at it as maximizing an expected reward over the future, but that sort of definition is pathological in various ways.

And my friend David Weinbaum, aka Weaver, he had a beautiful PhD thesis on open-ended intelligence, trying to conceive intelligence in a- Without a reward.

He's looking at complex self-organizing systems and looking at an intelligent system as being 1 that revises and grows and improves itself in conjunction with its environment without necessarily there being 1 objective function it's trying to maximize, although over certain intervals of time, it may act as if it's optimizing a certain objective function.

Very much Solaris from Stanislaw Lem's novels, right?

So yeah, the point is artificial general and intelligence.

On the other hand, everyone knows what AI is, and AGI seems immediately comprehensible to people with a technical background.

Now it's out there everywhere, which baffles me.

We're stuck with AGI probably for a very long time, until AGI systems take over and rename themselves.

too, which mostly have nothing to do with graphics anymore.

I wonder what the AGI system will call us humans.


Speaker 0: Maybe it's good to step back a little bit. I mean, we've been using the term AGI. People often cite you as the creator, or at least the popularizer of the term AGI, artificial general intelligence. Can you tell the origin story of the term?

Speaker 1: Sure, sure. So yeah, I would say

Speaker 2: I launched the term AGI upon the world for what it's worth without ever fully being in love with the term. What happened is I was editing a book, and this process started around 2001 or 2, I think the book came out 2005, if I'm not mistaken. I was editing a book which I provisionally was titling Real AI.

Speaker 1: And I

Speaker 2: mean, the goal was to gather together fairly serious academicish papers on the topic of making thinking machines that could really think in the sense like people can or even more broadly than people can, right? So then I was reaching out to other folks that I had encountered here or there who were interested in that, which included some other folks who I knew from the transhumanist and singularitarian world, like Peter Vos, who has a company, AGI Incorporated, still in California, and included Shane Legg, who had worked for me at my company WebMind in New York in the late 90s, who by now has become rich and famous. He was 1 of

Speaker 1: the co-founders of Google DeepMind. But

Speaker 2: at that time, Shane was, I think he may have just started doing his PhD with Markus Hutter, who at that time hadn't yet published his book, universal AI, which sort of gives a mathematical foundation for artificial general intelligence. So I reached out to Shane and Marcus and Peter Vos and Pei Wang, who was another former employee of mine who had been Douglas Hofstadter's PhD student, who had his own approach to AGI, and a bunch of some Russian folks reached out to these guys and they contributed papers for the book. But that was my provisional title, but I never loved it because in the end, I was doing some, what we would now call narrow AI as well, like applying machine learning to genomics data or chat data for sentiment analysis. I mean, that work is real. In a sense, it's really AI.

It's just a different kind of AI. Ray Kurzweil wrote about narrow AI versus strong AI. But that seemed weird to me because, first of all, narrow and strong are not antinodes. Right? That's right.

But secondly, strong AI was used in the cognitive science literature to mean the hypothesis that digital computer AIs could have true consciousness like human beings. So there was already a meaning to strong AI, which was complexly different but related. So we were tossing around on an email list whether what title it should be, and so we talked about narrow AI, broad AI, wide AI, narrow AI, general AI, and I think it was either Shane Legg or Peter Vos on the private email discussion we had, he said, well, why don't we go with AGI, artificial general intelligence. Pei Wang wanted to do GAI, General Artificial Intelligence, because in Chinese it goes in that order. But we figured gay wouldn't work in US culture at that time.

Speaker 1: So we went with the AGI, We used it for the title

Speaker 2: of that book. And part of Peter and Shane's reasoning was you have the G factor in psychology, which is IQ, general intelligence, right? So you have a meaning of GI, general intelligence in psychology, so then you're looking like artificial GI. So then we

Speaker 1: use that for the title of

Speaker 2: the book. And so I think, maybe both Shane and Peter think they invented the term, but then later after the book was published, this guy Mark Gubrid came up to me and he's like, well, I published an essay with the term AGI in like 1997 or something. And so I'm just waiting for some Russian to come out

Speaker 1: and say they published that in 1953. I mean, that term is not dramatically innovative or anything. It's 1 of these obvious in hindsight things, which is also

Speaker 2: annoying in a way, because, you know, Joe Chabac, who you interviewed, is a close friend of mine. He likes the term synthetic intelligence, which I like much better, but it hasn't actually caught on, right? Because, I mean,

Speaker 1: artificial is

Speaker 2: a bit off to me because Artifice is like a tool or something, but not all AGI's are gonna be tools. I mean, they may be now, but we're aiming toward making them agents rather than tools. And in a way, I don't like the distinction between artificial and natural, because I mean, we're part of nature also, and machines are part of nature. You can look at evolved versus engineered, but that's a different distinction. Then it should be engineered general intelligence.

And then general, well, if you look at Marcus Hutter's book, Universally, what he argues there is, within the domain of computation theory, which is limited but interesting, so if you assume computable environments and computable reward functions, then he articulates what would be a truly general intelligence,

Speaker 1: a system called AIXI, which is quite beautiful.

Speaker 0: AIXI.

Speaker 2: AIXI, and that's the middle name of my latest child, actually.

Speaker 0: What's the first name?

Speaker 2: First name is QRXI, Q-R-X-I, which my wife came up with, but that's an acronym for quantum organized rational expanding intelligence. And his middle name is Exiphanes, actually, which means the former principal underlying AXE. But in any

Speaker 0: case- You're giving Elon Musk's new child a run for his money.

Speaker 1: Well, I did it first. He copied me with this new freakish name. But now if I have another baby, I'm going to have to outdo him. It's becoming an arms race of weird, geeky baby names. We'll see what the babies think about it.

Speaker 2: But I mean, my oldest son, Zarathustra, loves his name, and my daughter, Sherazade, loves her name. So far, basically, if you

Speaker 1: give your kids weird names, then you're obliged to make the kids weird enough that they like the names, right? It directs their upbringing in a certain way. But yeah, anyway, I mean, what Marcus showed in that book is that a truly general intelligence theoretically is possible but would take infinite computing power. So then the artificial is a little off. The general is not really achievable within physics as we know it.

I mean, physics as we know it may be limited, but that's what we have to work with now. Intelligence.

Speaker 0: Infinitely general, you mean, like information processing perspective, yeah.

Speaker 2: Yeah, intelligence is not very well defined either. I mean, what does it mean? I mean, in AI now, it's fashionable to look at it as maximizing an expected reward over the future, but that sort of definition is pathological in various ways. And my friend David Weinbaum, aka Weaver, he had a beautiful PhD thesis on open-ended intelligence, trying to conceive intelligence in a- Without a reward. Without a death penalty.

Speaker 1: Yeah, he's just looking at

Speaker 2: it differently. He's looking at complex self-organizing systems and looking at an intelligent system as being 1 that revises and grows and improves itself in conjunction with its environment without necessarily there being 1 objective function it's trying to maximize, although over certain intervals of time, it may act as if it's optimizing a certain objective function. Very much Solaris from Stanislaw Lem's novels, right? So yeah, the point is artificial general and intelligence. They're all bad.

On the other hand, everyone knows what AI is, and AGI seems immediately comprehensible to people with a technical background. So I think that the term

Speaker 1: has served a sociological function. Now it's out there everywhere, which baffles me.

Speaker 0: It's like KFC, I mean, that's it. We're stuck with AGI probably for a very long time, until AGI systems take over and rename themselves. Yeah, and then we'll be biological.

Speaker 1: We're stuck with GPUs

Speaker 2: too, which mostly have nothing to do with graphics anymore.

Speaker 0: I wonder what the AGI system will call us humans. That was maybe.

Speaker 1: Grandpa. Yeah.

See all Lex Fridman transcripts on Youtube

Origin of the term AGI (Ben Goertzel) | AI Podcast Clips