Douglas Lenat is the founder of Cyc, a 37 year project aiming to solve common-sense knowledge and reasoning in AI. Please support this podcast by checking out our sponsors:
- Squarespace: https://lexfridman.com/squarespace and use code LEX to get 10% off
- BiOptimizers: http://www.magbreakthrough.com/lex to get 10% off
- Stamps.com: https://stamps.com and use code LEX to get free postage & scale
- LMNT: https://drinkLMNT.com/lex to get free sample pack
- ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free

EPISODE LINKS:
Douglas's Twitter: https://twitter.com/cycorpai
Cyc's Website: https://cyc.com

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
1:11 - What is Cyc?
9:17 - How to form a knowledge base of the universe
19:43 - How to train an AI knowledge base
24:04 - Global consistency versus local consistency
48:25 - Automated reasoning
54:05 - Direct uses of AI and machine learning
1:06:43 - The semantic web
1:17:16 - Tools to help Cyc interpret data 
1:26:26 - The most beautiful idea about Cyc
1:32:25 - Love and consciousness in AI
1:39:24 - The greatness of Marvin Minsky 
1:44:18 - Is Cyc just a beautiful dream?
1:49:03 - What is OpenCyc and how was it born?
1:54:53 - The open source community and OpenCyc
2:05:20 - The inference problem
2:07:03 - Cyc's programming language
2:14:37 - Ontological engineering
2:22:02 - Do machines think?
2:30:47 - Death and consciousness
2:40:48 - What would you say to AI?
2:45:24 - Advice to young people 
2:47:20 - Mortality

SOCIAL:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Reddit: https://reddit.com/r/lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

The following is a conversation with Doug Lennett, creator of Psyche, a system that for close to 40 years and still today has sought to solve the core problem of artificial intelligence, the acquisition of common sense knowledge and the use of that knowledge to think, to reason, and to understand the world.

To support this podcast, please check out our sponsors in the description.

As a side note, let me say that in the excitement of the modern era of machine learning, It is easy to forget just how little we understand exactly how to build the kind of intelligence that matches the power of the human mind.

To me, many of the core ideas behind psych, in some form, in actuality or in spirit, will likely be part of the AI system that achieves general super intelligence.

But perhaps more importantly, solving this problem of common sense knowledge will help us humans understand our own minds, the nature of truth, and finally, how to be more rational and more kind to each other.

This is the Lex Friedman Podcast and here is my conversation with Doug Lenat.

Psych is a project launched by you in 1984 and still is active today, Whose goal is to assemble a knowledge base that spans the basic concepts and rules about how the world works In other words, it hopes to capture common-sense knowledge Which is a lot harder than it sounds Can you elaborate on this mission and maybe perhaps speak to the various sub-goals within this mission?

When I was a faculty member in the computer science department at Stanford, my colleagues and I did research in all sorts of artificial intelligence programs.

So natural language understanding programs, robots, expert systems, and so on.

And we kept hitting the very same brick wall.

Our systems would have impressive early successes.

And so if your only goal was academic, namely to get enough material to write a journal article, that might actually suffice.

But if you're really trying to get AI, then you have to somehow get past the brick wall.

And the brick wall was the programs didn't have what we would call common sense.

They didn't have general world knowledge.

They didn't really understand what they were doing, what they were saying, what they were being asked.

And so very much like a clever dog performing tricks, we could get them to do tricks, but they never really understood what they were doing.

Sort of like when you get a dog to fetch your morning newspaper.

The dog might do that successfully, but the dog has no idea what a newspaper is or what it says or anything like that.

What does it mean to understand something?

Can you maybe elaborate on that a little bit?

Is it, is understanding action of like combining little things together like through inference or is understanding the wisdom you gain over time that forms a knowledge?

I think of understanding more like a, think of it more like the ground you stand on which could be very shaky, could be very unsafe, but most of the time is not because underneath it is more ground and eventually rock and other things.

But layer after layer after layer, that solid foundation is there.

And you rarely need to think about it, you rarely need to count on it, but occasionally you do.

And I've never used this analogy before, so bear with me.

But I think the same thing is true in terms of getting computers to understand things, which is you ask a computer a question, for instance, Alexa or some robot or something, And maybe it gets the right answer.

But if you were asking that of a human, you could also say things like, why?

And it might be a little annoying if you have a small child and they keep asking why questions in series.

Eventually you get to the point where you throw up your hands and say, I don't know, it's just the way the world is.

But for many layers, you actually have that layered solid foundation of support so that when you need it, you can count on it.

Well, when things are unexpected, when you come up against a situation which is novel, For instance, when you're driving, it may be fine to have a small program, a small set of rules that cover 99% of the cases, but that 1% of the time when something strange happens, you really need to draw on common sense.

For instance, my wife and I were driving recently and there was a trash truck in front of us.

And I guess they had packed it too full and the back exploded and Trash bags went everywhere.

And we had to make a split-second decision.

Are we going to swerve into another lane?

And in front of us was a large trash bag, and we know what we throw away in trash bags, probably not a safe thing to run over.

Over on the left was a bunch of fast food restaurant trash bags.

It's like, oh, well, those things are just like styrofoam and leftover food.

And so that was a safe thing for us to do.

Now that's the kind of thing that's gonna happen maybe once in your life.

But the point is that there's almost no telling what little bits of knowledge about the world you might actually need in some situations which were unforeseen.

But see, when you sit on that mountain or that ground that goes deep of knowledge in order to make a split second decision about fast food trash or random trash from the back of a trash truck, you need to be able to leverage that ground you stand on in some way.

It's not merely, you know, it's not enough to just have a lot of ground to stand on.

It's your ability to leverage it, to utilize it and like integrate it all together to make that split second decision.

And I suppose understanding isn't just having common sense knowledge to access, it's the act of accessing it somehow.

Like correctly filtering out the parts of the knowledge that are not useful, selecting only the useful parts, and effectively making conclusive decisions.

So let's tease apart 2 different tasks, really, both of which are incredibly important and even necessary If you're going to have this in a useful, usable fashion, as opposed to say, like library books sitting on a shelf and so on, where the knowledge might be there, but if a fire comes, the books are going to burn because they don't know what's in them and they're just going to sit there while they burn.

So there are 2 aspects of using the knowledge.

1 is a kind of a theoretical how is it possible at all?

And then the second aspect of what you said is, how can you do it quickly enough?

So how can you do it at all is something that philosophers have grappled with.

And fortunately, philosophers 100 years ago and even earlier developed a kind of formal language like English.

It's called predicate logic or first order logic or something like predicate calculus and so on.

So there's a way of representing things in this formal language which enables a mechanical procedure to sort of grind through and algorithmically produce all of the same logical entailments, all the same logical conclusions that you or I would from that same set of pieces of information that are represented that way.

So that sort of raises a couple questions.

1 is, how do you get all this information from say observations and English and so on into this logical form?

And secondly, how can you then efficiently run these algorithms to actually get the information you need in the case I mentioned in a 10th of a second, rather than say in 10 hours or 10,000 years of computation.

And like a corollary addition to the first 1 is how many such things do you need to gather for it to be useful in certain contexts?

So like what, in order, you mentioned philosophers, in order to capture this world and represent it in a logical way and with a formal logic, like how many statements are required?

That's, as far as I understand, is probably still an open question.

It may forever be an open question to say definitively about, to describe the universe perfectly.

I guess I'm gonna disappoint you by giving you an actual answer to your question.

So let's leave the efficiency question aside for now.

So how does all this information get represented in logical form so that these algorithms, resolution theorem proving and other algorithms can actually grind through all the logical consequences of what you said.

And that ties into your question about, well, how many of these things do you need?

Because if the answer is small enough, then by hand, you could write them out 1 at a time.

I held a meeting at Stanford, where I was a faculty member there, where We assembled about half a dozen of the smartest people I know, people like Alan Newell and Marvin Minsky and Alan Kay and a few others.

Because he liked your, he commented about your system, Yurisco, at the time.

So we have all these different smart people, and we came together to address the question that you raised, which is, if it's important to represent common sense knowledge and world knowledge in order for AIs to not be brittle, in order for AIs not to just have the veneer of intelligence.

Well, how many pieces of common sense, how many if-then rules, for instance, would we have to actually write in order to essentially cover what people expect perfect strangers to already know about the world.

And I expected there would be an enormous divergence of opinion and computation, but amazingly everyone got an answer which was around a million.

And 1 person got the answer by saying, well, look, you can only burn into human long-term memory a certain number of things per unit time, like maybe 1 every 30 seconds or something.

And other than that, it's just short-term memory and it flows away like water and so on.

So by the time you're say 10 years old or so, how many things could you possibly have burned into your long-term memory?

Another person went in a completely different direction and said, well, if you look at the number of words in a dictionary, not a whole dictionary, but for someone to essentially be considered to be fluent in a language.

And then about how many things about each word would you have to tell it?

Another person said, Well, let's actually look at 1 single short, one-volume desk encyclopedia article.

And so we'll look at what was like a four-paragraph article or something.

And If we were going to sit there and represent every single thing that was there, how many assertions or rules or statements would we have to write in this logical language?

And then multiply that by all of the number of articles that there were and so on.

So all of these estimates came out with a million.

And so if you do the math, it turns out that like, oh, well, then maybe in something like

person years, in 1 or 2 person centuries, we could actually get this written down by hand.

And a marvelous coincidence, an opportunity existed right at that point in time, the early 1980s, there was something called the Japanese fifth generation computing effort.

Japan had threatened to do in computing and AI and hardware what they had just finished doing in consumer electronics and the automotive industry, namely resting control away from the United States and more generally away from the West.

That's how you know it was a long time ago, because Congress did something.

Congress passed something called the National Cooperative Research Act, NCRA.

And what it said was, hey, all you big American companies, that's also how you know it was a long time ago because they were American companies rather than multinational companies.

Hey, all you big American companies, normally it would be an antitrust violation if you colluded on R&D.

But we promise for the next 10 years, we won't prosecute any of you if you do that to help combat this threat.

And so overnight, the first 2 consortia, research consortia in America sprang up, both of them coincidentally in Austin, Texas.

1 called Semitech focusing on hardware chips and so on.

And then 1 called MCC, the Microelectronics and Computer Technology Corporation, focusing more on software, on databases, and AI, and natural language understanding, and things like that.

And I got the opportunity, thanks to my friend Woody Bledsoe, who was 1 of the people who founded that, to come and be its principal scientist.

And he sent Admiral Bob Inman, who was the person running MCC, came and talked to me and said, look, Professor, you're talking about doing this project that's going to involve person-centuries of effort.

You've only got a handful of graduate students.

You do the math, it's going to take you longer than the rest of your life to finish this project.

But if you move to the wilds of Austin, Texas, we'll put 10 times as many people on it and you'll be done in a few years.

And the good news and bad news, the bad news is that all of us were off by an order of magnitude.

It turns out what you need are tens of millions of these pieces of knowledge about everyday, sort of like if you have a coffee cup with stuff in it and you turn it upside down, the stuff in it's gonna fall out.

So you need tens of millions of pieces of knowledge like that, Even if you take trouble to make each 1 as general as it possibly could be.

But the good news was that thanks to initially the fifth generation effort and then later U.S.

Government agency funding and so on, we were able to get enough funding not for a couple person centuries of time, but for a couple person millennia of time, which is what we've spent since

getting Psyche to contain the tens of millions of rules that it needs in order to really capture and span not all of human knowledge, but the things that you assume other people know, the things you count on other people knowing.

And the good news is since you've waited 38 years just about to talk to me, we're about at the end of that process.

So most of what we're doing now is not putting in even what you would consider common sense, but more putting in domain-specific applications, specific knowledge about healthcare in a certain hospital or about oil pipes getting clogged up or whatever the applications happen to be.

So we've almost come full circle and we're doing things very much like the expert systems of the 1970s and the 1980s, except instead of resting on nothing and being brittle, they're now resting on this massive pyramid, if you will, this massive lattice of common sense knowledge so that when things go wrong, when something unexpected happens, they can fall back on more and more and more general principles, eventually bottoming out in things like, for instance, if we have a problem with the microphone, 1 of the things you'll do is unplug it, plug it in again, and hope for the best, right?

Because that's 1 of the general pieces of knowledge you have in dealing with electronic equipment or software systems or things like that.

Like is it possible to encode something that generally captures this idea of turn it off and turn it back on and see if it fixes?

That's 1 of the things that that Psych knows.

That's actually 1 of the fundamental laws of nature.

So that basically covered the resources needed, and then we had to devise a method to actually figure out, well, what are the tens of millions of things that we need to tell the system.

And for that we found a few techniques which worked really well.

1 is to take any piece of text almost, it could be an advertisement, it could be a transcript, it could be a novel, it could be an article, And don't pay attention to the actual type that's there, the black space on the white page.

Pay attention to the complement of that, the white space, if you will.

So what did the writer of this sentence assume that the reader already knew about the world.

For instance, if they used a pronoun, how did they figure out that you—why did they think that you would be able to understand what the intended referent of that pronoun was?

If they used an ambiguous word, How did they think that you would be able to figure out what they meant by that word?

The other thing we look at is the gap between 1 sentence and the next 1.

What are all the things that the writer expected you to fill in and infer occurred between the end of 1 sentence and the beginning of the other.

So if the sentence says, Fred Smith robbed the Third National Bank, period.

He was sentenced to 20 years in prison, period.

Well, between the first sentence and the second, you're expected to infer things like Fred got caught, Fred got arrested, Fred went to jail, Fred had a trial, Fred was found guilty, and so on.

If my next sentence starts out with something like, the judge… then you assume it's the judge at his trial.

If my next sentence starts out something like, the arresting officer dot, dot, dot, you assume that it was the police officer who arrested him after he committed the crime and so on.

So those are 2 techniques for getting that knowledge.

The other thing we sometimes look at is fake news or humorous onion headlines or headlines in the Weekly World News, if you know what that is, or the National Enquirer, where it's like, oh, we don't believe this.

Then we introspect on why don't we believe it.

So there are things like B-17 lands on the Moon.

It's like, what do we know about the world that causes us to believe that that's just silly, or something like that?

Or Another thing we look for are contradictions, things which can't both be true.

And we say, what is it that we know that causes us to know that both of these can't be true at the same time?

For instance, in 1 of the weekly World News editions, in 1 article it talked about how Elvis was sighted, even though he was getting on in years and so on.

And another article in the same 1 talked about people seeing Elvis's ghost.

So it's like, why do we believe that at least 1 of these articles must be wrong?

So we have a series of techniques like that that enable our people.

And by now, we have about 50 people working full-time on this and have for decades.

So we've put in the thousands of person years of effort.

We've built up these tens of millions of rules.

We constantly police the system to make sure that we're saying things as generally as we possibly can.

So you don't want to say things like, no mouse is also a moose, because if you said things like that, then you'd have to add another 1 or 2 or 3 zeros onto the number of assertions you'd actually have to have.

So at some point, we generalize things more and more and we get to a point where we say, oh yeah, for any 2 biological taxons, if we don't know explicitly that 1 is a generalization of another, then almost certainly they're disjoint.

A member of 1 is not gonna be a member of the other, and so on, so.

And the same thing with the Elvis and the ghost, it has nothing to do with Elvis, it's more about human nature and the mortality and all that

Well, right, In general, things are not both alive and dead at the same time.

Yeah, unless special cats in theoretical physics examples.

Well, that raises a couple important points.

Well, that's the onion headline situation type of thing.

But no, no, So what you bring up is this really important point of like, well, how do you handle exceptions and inconsistencies and so on?

And 1 of the hardest lessons for us to learn, it took us about 5 years to really grit our teeth and learn to love it, is we had to give up global consistency.

So the knowledge base can no longer be consistent.

I grew up watching Star Trek, And anytime a computer was inconsistent, it would either freeze up or explode or take over the world or something bad would happen.

Or if you come from a mathematics background, once you can prove false, you can prove anything.

So that's why on the old knowledge-based systems were all very, very consistent.

But the trouble is that by and large, our models of the world, the way we talk about the world and so on, there are all sorts of inconsistencies that creep in here and there that will kill some attempt to build some enormous globally consistent knowledge base.

And so what we had to move to was a system of local consistency.

So a good analogy is you know that the surface of the Earth is more or less spherical globally, but you live your life every day as though the surface of the Earth were flat.

When you're talking to someone in Australia, you don't think of them as being oriented upside down to you when you're planning a trip.

Even if it's a thousand miles away, you may think a little bit about time zones, but you rarely think about the curvature of the earth and so on.

And for most purposes, you can live your whole life without really worrying about that because the Earth is locally flat.

In much the same way, the psych knowledge base is divided up into almost like tectonic plates, which are individual contexts.

And each context is more or less consistent, but there can be small inconsistencies at the boundary between 1 context and the next 1 and so on.

And so, by the time you move, say, 20 contexts over, there could be glaring inconsistencies.

So eventually you get from the normal modern real-world context that we're in right now to something like Roadrunner cartoon context where physics is very different.

And in fact, life and death are very different because no matter how many times he's killed, the coyote comes back in the next scene and so on.

So that was a hard lesson to learn and we had to make sure that our representation language, the way that we actually encode the knowledge and represent it, was expressive enough that we could talk about things being true in 1 context and false in another.

Things that are true at 1 time and false in another.

Things that are true, let's say, in 1 region, like 1 country, but false in another.

Things that are true in 1 person's belief system, but false in another person's belief system.

Things that are true at 1 level of abstraction and false at another.

For instance, at 1 level of abstraction, you think of this table as a solid object, but down at the atomic level, it's mostly empty space and so on.

So then that's fascinating, but it puts a lot of pressure on context to do a lot of work.

So you say tectonic plates, is it possible to formulate contexts that are general and big that do this kind of capture of knowledge bases?

Or do you then get turtles on top of turtles again where there's just a huge number of contexts?

So, it's good you asked that question because you're pointed in the right direction which is you want context to be first class objects in your system's knowledge base, in particular in Psyche's knowledge base.

And by first-class object, I mean that we should be able to have Psyche think about and talk about and reason about 1 context or another context the same way it reasons about coffee cups and tables and people and fishing and so on.

And so, contexts are just terms in its language, just like the ones I mentioned.

And so, Psyche can reason about context, context can arrange hierarchically, and so on.

And so, you can say things about, let's say, things that are true in the modern era, things that are true in a particular year would then be a sub-context of the things that are true in a broad, let's say, a century or a millennium or something like that.

Things that are true in Austin, Texas are generally going to be a specialization of things that are true in Texas, which is going to be a specialization of things that are true in Texas, which is gonna be a specialization of things that are true in the United States, and so on.

And so you don't have to say things over and over again at all these levels, you just say things at the most general level that it applies to, and you only have to say it once, and then it essentially inherits to all these more specific contexts.

To ask a slightly technical question, is this inheritance a tree or a graph?

Oh, you definitely have to think of it as a graph.

So we could talk about, for instance, why the Japanese fifth-generation computing effort failed.

There were about half a dozen different reasons.

1 of the reasons they failed was because they tried to represent knowledge as a tree rather than as a graph.

And so each node in their representation could only have 1 parent node.

So if you had a table that was a wooden object, a black object, a flat object, and so on.

You have to choose 1, and that's the only parent it could have.

When, of course, depending on what it is you need to reason about it, sometimes it's important to know that it's made out of wood, like if we're talking about a fire.

Sometimes it's important to know that it's flat if we're talking about resting something on it and so on.

So 1 of the problems was that they wanted a kind of Dewey decimal numbering system for all of their concepts, which meant that each node could only have at most 10 children, and each node could only have 1 parent.

And while that does enable the Dewey Decimal type numbering of concepts, labeling of concepts, it prevents you from representing all the things you need to about objects in our world.

And that was 1 of the things which they never were able to overcome, and I think that was 1 of the main reasons that that project failed.

So we'll return to some of the doors you've opened, but if we can go back to that room in 1984, around there with Marvin Minsky and Stanford.

By the way, I should mention that Marvin wouldn't do his estimate until someone brought him an envelope so that he could literally do a back of the envelope calculation to come up with his number.

Well, because I feel like the conversation in that room is an important 1.

A few people get together and plant the seed of ideas and they reverberate throughout history and some kind of dissipate and disappear and some, you know, Drake equation and you know, it seems like a meaningless equation, somewhat meaningless, but I think it drives and motivates a lot of scientists.

And when the aliens finally show up, that equation will get even more valuable because then we'll be able to, in the long arc of history, the Drake equation will prove to be quite useful, I think.

See all Lex Fridman transcripts on Youtube

Douglas Lenat: Cyc and the Quest to Solve Common Sense Reasoning in AI | Lex Fridman Podcast #221