The following is a conversation with David Ferrucci.

He led the team that built Watson, the IBM question answering system that beat the top humans in the world at the game of Jeopardy.

For spending a couple hours with David, I saw a genuine passion, not only for abstract understanding of intelligence, but for engineering it to solve real world problems under real world deadlines and resource constraints.

Where science meets engineering is where brilliant simple ingenuity emerges.

People who work at joining it too have a lot of wisdom earned through failures and eventual success.

David is also the founder, CEO, and chief scientist of Elemental Cognition, a company working to engineer AI systems that understand the world the way people do.

This is the Artificial Intelligence Podcast.

If you enjoy it, subscribe on YouTube, give it 5 stars on iTunes, support it on Patreon, or simply connect with me on Twitter, Alex Friedman, spelled F-R-I-D-M-A-N.

And now, here's my conversation with David Ferrucci.

Your undergrad was in biology with an eye toward medical school before you went on for the PhD in computer science.

What is the difference between biological systems and computer systems?

When you sit back, look at the stars, and think philosophically?

I often wonder whether or not there is a substantive difference.

I mean, I think the thing that got me into computer science, mental artificial intelligence was exactly this presupposition that if we can get machines to think, or I should say this question, this philosophical question, If we can get machines to think, to understand, to process information the way we do, so if we can describe a procedure, describe a process, even if that process were the intelligence process itself, then what would be the difference?

So from a philosophical standpoint, I'm not sure I'm convinced that there is.

I mean, you can go in the direction of spirituality, you can go in the direction of a soul, but in terms of what we can experience from an intellectual and physical perspective, I'm not sure there is.

Clearly, there are different implementations, But if you were to say, is a biological information processing system fundamentally more capable than 1 we might be able to build out of silicon or some other substrate, I don't know that there is.

How distant do you think is the biological implementation?

So fundamentally they may have the same capabilities, but is it really a far mystery where a huge number of breakthroughs are needed to be able to understand it, or is it something that, for the most part, in the important aspects, echoes of the same kind of characteristics?

I mean, so, you know, your question presupposes that there's this goal to recreate, you know, what we perceive as biological intelligence.

I'm not sure that's the, I'm not sure that's how I would state the goal.

I think that understanding the human brain and how it works is important for us to be able to diagnose and treat issues for us to understand our own strengths and weaknesses, both intellectual, psychological, and physical.

So neuroscience and understanding the brain from that perspective, there's a clear goal there.

From the perspective of saying, I wanna mimic human intelligence, That one's a little bit more interesting.

Human intelligence certainly has a lot of things we envy.

So I think we're capable of sort of stepping back and saying what do we want out of an intelligence?

How do we want to communicate with that intelligence?

Now, of course, it's somewhat of an interesting argument because I'm sitting here as a human with a biological brain and I'm critiquing the strengths and weaknesses of human intelligence and saying that we have the capacity to step back and say, gee, what is intelligence and what do we really want out of it, and that in and of itself suggests that human intelligence is something quite enviable, that it can introspect that way.

Yeah, I think that flaws that human intelligence has is extremely prejudicial and biased in the way it draws many inferences.

Do you think those are, sorry to interrupt, do you think those are features or are those bugs?

Do you think the prejudice, the forgetfulness, the fear, what are the flaws?

You think those are all things that can be get in the way of intelligence or the essential components of intelligence?

Well again, if you go back and you define intelligence as being able to sort of accurately, precisely, rigorously reason, develop answers and justify those answers in an objective way, yeah then human intelligence has these flaws in that it tends to be more influenced by some of the things you said, and it's largely an inductive process, meaning it takes past data, uses that to predict the future, very advantageous in some cases, but fundamentally biased and prejudicial in other cases, because it's gonna be strongly influenced by its priors, whether they're right or wrong from some objective reasoning perspective, you're gonna favor them because those are the decisions or those are the paths that succeeded in the past.

And I think that mode of intelligence makes a lot of sense for when your primary goal is to act quickly and survive and make fast decisions.

And I think those create problems when you want to think more deeply and make more objective and reasoned decisions.

They do sort of 1 more naturally than they do the other, but they're capable of doing both.

You're saying they do the 1 that responds quickly more naturally.

Because that's the thing we kind of need to not be eaten by

For example, but then we've learned to reason through logic, we've developed science, we've trained people to do that.

I think that's harder for the individual to do.

I think it requires training and teaching, I think we are, the human mind certainly is capable of it, but we find it more difficult.

And then there are other weaknesses, if you will, as you mentioned earlier, just memory capacity and how many chains of inference can you actually go through without losing your way, so just focus.

So the way you think about intelligence, and we're really sort of floating in this philosophical space, but I think you're the perfect person to talk about this, because we'll get to Jeopardy and beyond.

That's 1 of the most incredible accomplishments in AI, in the history of AI, but hence the philosophical discussion.

So let me ask, you've kind of alluded to it, but let me ask again, what is intelligence?

Underlying the discussion we'll have with Jeopardy and beyond, how do you think about intelligence?

Is it a sufficiently complicated problem, being able to reason your way through solving that problem?

Is that kind of how you think about what it means to be intelligent?

So I think of intelligence primarily 2 ways.

So in other words, if I have a problem, can I predict what's gonna happen next?

Whether it's to predict the answer of a question or to say, look, I'm looking at all the market dynamics and I'm gonna tell you what's gonna happen next.

Or you're in a room and somebody walks in and you're going to predict what they're going to do next or what they're going to say next.

So in a highly dynamic environment full of uncertainty, be able to predict.

you know, the more variables, the more complex, the more possibilities, the more complex.

But can I take a small amount of prior data and learn the pattern and then predict what's gonna happen next accurately and consistently?

You need to have an understanding of the way the world works in order to be able to unroll it into the future, right?

This is very much what deep learning does, machine learning does, is if you give me enough prior data and you tell me what the output variable is that matters, I'm gonna sit there and be able to predict it.

And if I can predict it accurately so that I can get it right more often than not.

I'm smart, if I do that with less data and less training time, I'm even smarter.

If I can figure out what's even worth predicting, I'm smarter, meaning I'm figuring out what path is gonna get me toward a goal.

Well, that's interesting about picking a goal, sort of an interesting thing.

I think that's where you bring in, what are you pre-programmed to do?

We talk about humans, and humans are pre-programmed to survive, so sort of their primary driving goal.

So it's not just figuring out that you need to run away from the ferocious tiger, but we survive in a social context as an example.

So understanding the subtleties of social dynamics becomes something that's important for surviving, finding a mate, reproducing, right?

So we're continually challenged with complex sets of variables, complex constraints, rules, if you will, or patterns, and we learn how to find the functions and predict the things, in other words, represent those patterns efficiently and be able to predict what's gonna happen.

That doesn't really require anything specific other than the ability to find that function and predict that right answer.

But then when we say, well, do we understand each other?

In other words, would you perceive me as intelligent beyond that ability to predict?

So now I can predict, but I can't really articulate how I'm going to that process, what my underlying theory is for predicting, and I can't get you to understand what I'm doing so that you can follow, you can figure out how to do this yourself if you did not have, for example, the right pattern-managing machinery that I did.

Now we potentially have this breakdown where in effect I'm intelligent, but I'm sort of an alien intelligence relative to you.

You're intelligent but nobody knows about it.

So you're saying, let's sort of separate the 2 things.

1 is you explaining why you were able to predict the future.

And the second is me being able to, impressing me that you're intelligent.

Me being able to know that you successfully predicted the future.

Well, It's not impressing you that I'm intelligent.

In other words, you may be convinced that I'm intelligent in some form.

But then what happens is, if I say, how are you doing that?

And you can't communicate with me, and you can't describe that to me, now I may label you a savant.

I may say, well, you're doing something weird and it's just not very interesting to me because you and I can't really communicate.

And so now, so this is interesting, right?

Because now this is, you're in this weird place where for you to be recognized as intelligent the way I'm intelligent, then you and I sort of have to be able to communicate.

And then we start to understand each other, and then my respect and my appreciation, my ability to relate to you starts to change.

So now you're not an alien intelligence anymore, you're a human intelligence now because you and I can communicate.

And so I think when we look at animals, for example, animals can do things we can't quite comprehend, we don't quite know how they do them, but they can't really communicate with us.

They can't put what they're going through in our terms.

And so we think of them as sort of, well, they're these alien intelligences and they're not really worth necessarily what we're worth.

We don't treat them the same way as a result of that.

But it's hard because who knows what's going on.

The explaining that you're intelligent, explaining the reasoning that went into the prediction is not some kind of mathematical proof.

If we look at humans, look at political debates and discourse on Twitter, it's mostly just telling stories.

So your task is, sorry, your task is not to tell an accurate depiction of how you reason, but to tell a story, real or not, that convinces me that there was a mechanism by which you

I mean, even a mathematical proof is that.

Because ultimately, the other mathematicians have to be convinced by your proof.

Otherwise, in fact, there have been several.

Yeah, there have been several proofs out there where mathematicians would study it for a long time before they were convinced that it actually proved anything, right?

You never know if it proved anything until the community mathematicians decided that it did.

But it's a real thing, and that's sort of the point, right, is that ultimately, this notion of understanding, us understanding something is ultimately a social concept.

In other words, I have to convince enough people that I did this in a reasonable way.

I did this in a way that other people can understand and replicate and that it makes sense to them.

So human intelligence is bound together in that way.

We sort of never really get away with it until we can sort of convince others that our thinking process makes sense.

Did you think the general question of intelligence is then also a social construct?

So if we ask questions of an artificial intelligence system, is this system intelligent?

The answer will ultimately be a socially constructed concept.

I'm saying we can try to define intelligence in this super objective way that says, here's this data, I wanna predict this type of thing, learn this function, and then if you get it right, often enough, we consider you intelligent.

I think it is, it doesn't mean it's not useful.

It could be solving a problem we can't otherwise solve and can solve it more reliably than we can.

But then there's this notion of, can humans take responsibility for the decision that you're making?

Can we relate to the process that you're going through?

And now you as an agent, whether you're a machine or another human, frankly, are now obliged to make me understand how it is that you're arriving at that answer and allow me, me or obviously a community or a judge of people to decide whether or not that makes sense.

And by the way, that happens with humans as well.

You're sitting down with your staff, for example, and you ask for suggestions about what to do next, and someone says, oh, I think you should buy, and I think you should buy this much, or whatever, or sell, or whatever it is, or I think you should launch the product today or tomorrow, or launch this product versus that product, whatever the decision may be, and you ask why, and the person says, I just have a good feeling about it, and you're not very satisfied.

Now, that person could be, you might say, well, you've been right before, but I'm gonna put the company on the line.

Can you explain to me why I should believe this?

And that explanation may have nothing to do with the truth.

And that's why I'm saying it's, we're bound together.

Right, our intelligences are bound together in that sense.

And if, for example, you're giving me an explanation, I mean, this is a very important point, right?

You're giving me an explanation, I'm not good, and then I'm not good at reasoning well, and being objective, and following logical paths and consistent paths, and I'm not good at measuring and sort of computing probabilities across those paths.

What happens is collectively, we're not gonna do well.

So I think we'll talk quite a bit about the first on a specific objective metric benchmark performing well.

But being able to explain the steps, the reasoning,

I mean, I think that that's, Well, it's hard for humans.

The thing that's hard for humans, as you know, may not necessarily be hard for computers and vice versa.

So, sorry, so how hard is that problem for computers?

I think it's hard for computers and the reason why I related to or saying that it's also hard for humans is because I think when we step back and we say we want to design computers to do that, 1 of the things we have to recognize is we're not sure how to do it well.

And even if you wanted to learn it, it's not clear exactly what data we use and what judgments we use to learn that well.

And so what I mean by that is, if you look at the entire enterprise of science, science is supposed to be about objective reason and reason, right?

So we think about, gee, who's the most intelligent person or group of people in the world, do we think about the savants who can close their eyes and give you a number?

We think about the think tanks or the scientists or the philosophers who kind of work through the details and write the papers and come up with the thoughtful, logical proves and use the scientific method.

And my point is that how do you train someone to do that?

What's the process of training people to do that well?

We work, as a society, we work pretty hard to get other people to understand our thinking and to convince them of things.

Now we could persuade them, obviously we talked about this, like human flaws or weaknesses, we can persuade them through emotional means, but to get them to understand and connect to and follow a logical argument is difficult.

We try, we do it as scientists, we try to do it as journalists, we try to do it as even artists in many forms, as writers, as teachers.

We go through a fairly significant training process to do that.

And then we could ask, well, why is that so hard?

But it's hard, and for humans it takes a lot of work.

And when we step back and say, well, how do we get a machine to do that?

How would you begin to try to solve that?

And maybe just a quick pause, because there's an optimistic notion in the things you're describing, which is being able to explain something through reason.

But if you look at algorithms that recommend things that we'll look at next, whether it's Facebook, Google, advertisement-based companies, you know, their goal is to convince you to buy things based on anything.

So that could be reason, because the best of advertisement is showing you things that you really do need and explain why you need it.

But it could also be through emotional manipulation.

The algorithm that describes why a certain reason, a certain decision was made, how hard is it to do it through emotional manipulation?

So you've kind of focused on reason, logic, really showing in a clear way why something is good.

1, is that even a thing that us humans do?

And 2, how do you think of the difference in the reasoning aspect and the emotional manipulation?

So you call it emotional manipulation, but more objectively it's essentially saying, there are certain features of things that seem to attract your attention.

I mean, it kind of give you more of that stuff.

Yeah, I mean, I'm not saying it's right or wrong.

It works to get your attention and it works to get you to buy stuff.

And when you think about algorithms that look at the patterns of features that you seem to be spending your money on and say, I'm gonna give you something with a similar pattern, so I'm gonna learn that function because the objective is to get you to click on it or get you to buy it or whatever it is.

I guess this seems to be very useful for convincing,

For convincing humans, it's good, because again, this goes back to, what is the human behavior like?

How does the human brain respond to things?

I think there's a more optimistic view of that too, which is that if you're searching for certain kinds of things, you've already reasoned that you need them.

And these algorithms are saying, look, that's up to you to reason whether you need something or not, that's your job.

You may have an unhealthy addiction to this stuff or you may have a reasoned and thoughtful explanation for why it's important to you and the algorithms are saying, hey, that's like whatever, like that's your problem.

All I know is you're buying stuff like that, you're interested in stuff like that, could be a bad reason, could be a good reason, that's up to you, I'm gonna show you more of that stuff.

And so, and I think that that's, it's not good or bad, it's not reasoned or not reasoned, the algorithm is doing what it does, which is saying you seem to be interested in this, I'm gonna show you more of that stuff.

And I think we're seeing this not just in buying stuff, but even in social media, you're reading this kind of stuff, I'm not judging on whether it's good or bad, I'm not reasoning at all.

I'm just saying I'm gonna show you other stuff with similar features.

And that's it, and I wash my hands from it, and I say that's all that's going on.

So 1, the bar of performance is extremely high, and yet we also ask them to, in the case of social media, to help find the better angels of our nature and help make a better society.

What do you think about the role of AI there?

I agree with you, that's the interesting dichotomy, right, because on 1 hand, we're sitting there and we're sort of doing the easy part, which is finding the patterns.

We're not building, the system's not building a theory that is consumable and understandable by other humans that can be explained and justified.

And so on 1 hand to say, oh, you know, AI is doing this, why isn't it doing this other thing?

And it's interesting to think about why it's harder.

And because you're interpreting the data in the context of prior models, in other words, understandings of what's important in the world, what's not important, What are all the other abstract features that drive our decision making?

What's sensible, what's not sensible, what's good, what's bad, what's moral, what's valuable, what isn't?

So when I see you clicking on a bunch of stuff and I look at these simple features, the raw features, the features that are there in the data, like what words are being used or how long the material is or other very superficial features, what colors are being used in the material.

Like I don't know why you're clicking on the stuff you're clicking, or if it's products, what the price is, or what the category is, and stuff like that.

And I just feed you more of the same stuff.

That's very different than kind of getting in there and saying, What does this mean?

The stuff you're reading, why are you reading it?

What assumptions are you bringing to the table?

Does it lead you to thoughtful, good conclusions?

Again, there's interpretation and judgment involved in that process that isn't really happening in the AI today.

That's harder Because you have to start getting at the meaning of the stuff, of the content.

You have to get at how humans interpret the content relative to their value system and deeper thought processes.

So that's what meaning means, is not just some kind of deep, timeless, semantic thing that the statement represents but also how a large number of people are likely to interpret.

So it's again, even meaning is a social construct.

So you have to try to predict how most people would understand this kind of statement.

Yeah, meaning is often relative, but meaning implies that the connections go beneath the surface of the artifacts.

If I show you a painting, it's a bunch of colors on a canvas, what does it mean to you?

And it may mean different things to different people because of their different experiences.

It may mean something even different to the artist who painted it.

As we try to get more rigorous with our communication, we try to really nail down that meaning, so we go from abstract art to precise mathematics, precise engineering drawings and things like that, we're really trying to say, I wanna narrow that space of possible interpretations because the precision of the communication ends up becoming more and more important.

And so that means that I have to specify, and I think that's why this becomes really hard Because if I'm just showing you an artifact and you're looking at it superficially whether it's a bunch of words on a page or whether it's You know brushstrokes on a canvas or pixels in a photograph, you can sit there and you can interpret lots of different ways at many, many different levels.

But when I wanna align our understanding of that, I have to specify a lot more stuff that's actually not directly in the artifact.

Now I have to say, well, how are you interpreting this image and that image, and what about the colors, and what do they mean to you?

What perspective are you bringing to the table?

What are your prior experiences with those artifacts?

What are your fundamental assumptions and values?

What is your ability to kind of reason to chain together logical implication as you're sitting there and saying, well, if this is the case, then I would conclude this.

And if that's the case, then I would conclude this, and if that's the case, then I would conclude that.

So your reasoning processes and how they work, your prior models and what they are, your values and your assumptions, all those things now come together into the interpretation.

See all Lex Fridman transcripts on Youtube

David Ferrucci: IBM Watson, Jeopardy & Deep Conversations with AI | Lex Fridman Podcast #44