Full episode with Jim Keller (Feb 2020): https://www.youtube.com/watch?v=Nb2tebYAaOA
Clips channel (Lex Clips): https://www.youtube.com/lexclips
Main channel (Lex Fridman): https://www.youtube.com/lexfridman
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on Apple Podcasts (iTunes):
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Jim Keller is a legendary microprocessor engineer, having worked at AMD, Apple, Tesla, and now Intel. He's known for his work on the AMD K7, K8, K12 and Zen microarchitectures, Apple A4, A5 processors, and co-author of the specifications for the x86-64 instruction set and HyperTransport interconnect.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

So what have you learned about the human abstractions from individual functional human units to the broader organization?

What does it take to create something special?

Well, most people don't think simple enough.

All right, so do you know the difference between a recipe and the understanding?

There's probably a philosophical description of this.

So imagine you're gonna make a loaf of bread.

The recipe says get some flour, add some water, add some yeast, mix it up, let it rise, put it in a pan, put it in the oven.

Understanding bread, you can understand biology, supply chains, grain grinders, yeast, physics, thermodynamics, there's so many levels of understanding there.

And then when people build and design things, they frequently are executing some stack of recipes.

And the problem with that is the recipes all have limited scope.

Like if you have a really good recipe book for making bread, it won't tell you anything about how to make an omelet.

But if you have a deep understanding of cooking, right, than bread, omelets, you know, sandwich, you know, there's a different way of viewing everything.

And most people, when you get to be an expert at something, you're hoping to achieve deeper understanding, not just a large set of recipes to go execute.

And it's interesting to walk groups of people because executing recipes is unbelievably efficient if it's what you want to do.

If it's not what you want to do, you're really stuck.

And everybody has a balance of, let's say, deeper understanding recipes, and some people are really good at recognizing when the problem is to understand something deeply.

Does every stage of development, deep understanding on the team needed?

Well this goes back to the art versus science question.

If you constantly unpack everything for deeper understanding, you never get anything done.

And if you don't unpack understanding when you need to, you'll do the wrong thing.

And then at every juncture, like human beings are these really weird things because everything you tell them has a million possible outputs, right?

And then they all interact in a hilarious way.

And then having some intuition about what you tell them, what you do, when do you intervene, when do you not, it's complicated.

It's essentially computationally unsolvable.

But with deep understanding, do you mean also sort of fundamental questions of things like what is a computer?

Or why, like the why questions, why are we even building this?

Like of purpose, or do you mean more like going towards the fundamental limits of physics, sort of really getting into the core of the science?

Well, in terms of building a computer, Think a little simpler.

So common practices, you build a computer, and then when somebody says, I want to make it 10 percent faster, you'll go in and say, all right, I need to make this buffer bigger, and maybe I'll add an add unit.

Or I have this thing that's 3 instructions wide, I'm going to make it 4 instructions wide.

And what you see is each piece gets incrementally more complicated, right?

And then at some point you hit this limit, like adding another feature or buffer doesn't seem to make it any faster.

And then people will say, well that's because it's a fundamental limit.

And then somebody else will look at it and say, well actually the way you divided the problem up and the way that different features are interacting is limiting you and it has to be rethought, rewritten.

So then you refactor it and rewrite it and what people commonly find is the rewrite is not only faster but half as complicated.

So how often in your career, but just have you seen as needed, maybe more generally, to just throw the whole thing out

This is where I'm on 1 end of it, every 3 to 5 years.

If you want to really make a lot of progress on computer architecture, every 5 years, you should do 1 from scratch.

So where does the x86-64 standard come in?

I wrote the, I was the co-author of that spec in 98.

The instruction set itself has been extended quite a few times.

And instruction sets are less interesting than the implementation underneath.

There's been, on x86 architecture, Intel's designed a few, AIM's designed a few, very different architectures.

And I don't wanna go into too much of the detail about how often, but There's a tendency to rewrite it every 10 years, and it really should be every 5.

So you're saying you're an outlier in that sense.

To everybody involved, because like you said, repeating the recipe is efficient.

Companies want to make money, no, individual engineers want to succeed, so you want to incrementally improve, increase the buffer from 3 to 4.

is where you get into diminishing return curves.

So every, you have a project, and you start here, and it goes up, and you have diminishing return.

And to get to the next level, you have to do a new 1, and the initial starting point will be lower than the old optimization point, but it'll get higher.

So now you have 2 kinds of fear, short-term disaster and long-term disaster.

Like, you know, people with a quarter by quarter business objective are terrified about changing everything.

And people who are trying to run a business or build a computer for a long-term objective know that the short-term limitations block them from the long-term success.

So if you look at leaders of companies that had really good long-term success, every time they saw that they had to redo something, they did.

Like you optimize the old 1 while you build a new 1.

But the marketing guys are always like, promise me that the new computer is faster on every single thing.

And the computer architect says, well, the new computer will be faster on the average.

But there's a distribution of results and performance, and you'll have some outliers that are slower.

And that's very hard, because they have 1 customer who cares about that 1.


Speaker 0: So what have you learned about the human abstractions from individual functional human units to the broader organization? What does it take to create something special?

Speaker 1: Well, most people don't think simple enough. All right, so do you know the difference between a recipe and the understanding? There's probably a philosophical description of this. So imagine you're gonna make a loaf of bread. The recipe says get some flour, add some water, add some yeast, mix it up, let it rise, put it in a pan, put it in the oven.

It's a recipe. Understanding bread, you can understand biology, supply chains, grain grinders, yeast, physics, thermodynamics, there's so many levels of understanding there. And then when people build and design things, they frequently are executing some stack of recipes. And the problem with that is the recipes all have limited scope. Like if you have a really good recipe book for making bread, it won't tell you anything about how to make an omelet.

But if you have a deep understanding of cooking, right, than bread, omelets, you know, sandwich, you know, there's a different way of viewing everything. And most people, when you get to be an expert at something, you're hoping to achieve deeper understanding, not just a large set of recipes to go execute. And it's interesting to walk groups of people because executing recipes is unbelievably efficient if it's what you want to do. If it's not what you want to do, you're really stuck. And that difference is crucial.

And everybody has a balance of, let's say, deeper understanding recipes, and some people are really good at recognizing when the problem is to understand something deeply. Does that make sense?

Speaker 0: It totally makes sense. Does every stage of development, deep understanding on the team needed?

Speaker 1: Well this goes back to the art versus science question.

Speaker 0: Sure.

Speaker 1: If you constantly unpack everything for deeper understanding, you never get anything done. Right. And if you don't unpack understanding when you need to, you'll do the wrong thing. And then at every juncture, like human beings are these really weird things because everything you tell them has a million possible outputs, right? And then they all interact in a hilarious way.

And then having some intuition about what you tell them, what you do, when do you intervene, when do you not, it's complicated.

Speaker 0: Right, so.

Speaker 1: It's essentially computationally unsolvable.

Speaker 0: Yeah, it's an intractable problem, sure. Humans are a mess. But with deep understanding, do you mean also sort of fundamental questions of things like what is a computer? Or why, like the why questions, why are we even building this? Like of purpose, or do you mean more like going towards the fundamental limits of physics, sort of really getting into the core of the science?

Speaker 1: Well, in terms of building a computer, Think a little simpler. So common practices, you build a computer, and then when somebody says, I want to make it 10 percent faster, you'll go in and say, all right, I need to make this buffer bigger, and maybe I'll add an add unit. Or I have this thing that's 3 instructions wide, I'm going to make it 4 instructions wide. And what you see is each piece gets incrementally more complicated, right? And then at some point you hit this limit, like adding another feature or buffer doesn't seem to make it any faster.

And then people will say, well that's because it's a fundamental limit. And then somebody else will look at it and say, well actually the way you divided the problem up and the way that different features are interacting is limiting you and it has to be rethought, rewritten. Right? So then you refactor it and rewrite it and what people commonly find is the rewrite is not only faster but half as complicated. From scratch?

Speaker 0: Yes. So how often in your career, but just have you seen as needed, maybe more generally, to just throw the whole thing out

Speaker 1: and start over. This is where I'm on 1 end of it, every 3 to 5 years.

Speaker 0: Which end are you on? Rewrite more often. Rewrite, and 3 to 5 years is?

Speaker 1: If you want to really make a lot of progress on computer architecture, every 5 years, you should do 1 from scratch.

Speaker 0: So where does the x86-64 standard come in? Or what, how often do you?

Speaker 1: I wrote the, I was the co-author of that spec in 98. That's 20 years ago.

Speaker 0: Yeah, so that's still around.

Speaker 1: The instruction set itself has been extended quite a few times. Yes. And instruction sets are less interesting than the implementation underneath. There's been, on x86 architecture, Intel's designed a few, AIM's designed a few, very different architectures. And I don't wanna go into too much of the detail about how often, but There's a tendency to rewrite it every 10 years, and it really should be every 5.

Speaker 0: So you're saying you're an outlier in that sense. Rewrite more often. Rewrite more often.

Speaker 1: Well, and here's the problem.

Speaker 0: Isn't that scary?

Speaker 1: Yeah, of course. Well, scary to who?

Speaker 0: To everybody involved, because like you said, repeating the recipe is efficient. Companies want to make money, no, individual engineers want to succeed, so you want to incrementally improve, increase the buffer from 3 to 4. Well, increase performance. So this

Speaker 1: is where you get into diminishing return curves. I think Steve Jobs said this, right? So every, you have a project, and you start here, and it goes up, and you have diminishing return. And to get to the next level, you have to do a new 1, and the initial starting point will be lower than the old optimization point, but it'll get higher. So now you have 2 kinds of fear, short-term disaster and long-term disaster.

Speaker 0: And you're haunted. So grown-ups,

Speaker 1: Right? Like, you know, people with a quarter by quarter business objective are terrified about changing everything. And people who are trying to run a business or build a computer for a long-term objective know that the short-term limitations block them from the long-term success. So if you look at leaders of companies that had really good long-term success, every time they saw that they had to redo something, they did.

Speaker 0: And so somebody has to speak up.

Speaker 1: Or you do multiple projects in parallel. Like you optimize the old 1 while you build a new 1. But the marketing guys are always like, promise me that the new computer is faster on every single thing. And the computer architect says, well, the new computer will be faster on the average. But there's a distribution of results and performance, and you'll have some outliers that are slower.

And that's very hard, because they have 1 customer who cares about that 1.

See all Lex Fridman transcripts on Youtube

Jim Keller: Most People Don't Think Simple Enough | AI Podcast Clips