Full episode with Donald Knuth (Dec 2019): https://www.youtube.com/watch?v=2BdBfsXbST8
Clips channel (Lex Clips): https://www.youtube.com/lexclips
Main channel (Lex Fridman): https://www.youtube.com/lexfridman
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on Apple Podcasts (iTunes):
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Donald Knuth is one of the greatest and most impactful computer scientists and mathematicians ever. He is the recipient in 1974 of the Turing Award, considered the Nobel Prize of computing. He is the author of the multi-volume work, the magnum opus, The Art of Computer Programming. He made several key contributions to the rigorous analysis of the computational complexity of algorithms. He popularized asymptotic notation, that we all affectionately know as the big-O notation. He also created the TeX typesetting which most computer scientists, physicists, mathematicians, and scientists and engineers use to write technical papers and make them look beautiful.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

What's your thinking and writing process like every day?

Yeah, I guess it's actually the best question because I spend 7 days a week doing

Yeah, but okay, so the chair I'm sitting in is where I do...

My chair is usually sitting over there where I have other books, some reference books.

But I found this chair which was designed by a Swedish guy anyway.

It turns out this is the only chair I can really sit in for hours and hours and not know that I'm in a chair But then I have the stand-up desk right next next to us and and so after I write something with pencil and eraser I get up and I type it and revise and rewrite.

The kernel of the idea is first put on paper.

Right, and I'll write maybe 5 programs a week, of course, literate programming.

And these are, Before I describe something in my book, I always program it to see how it's working, and I try it a lot.

For example, at the end of January, I learned of a breakthrough by 4 Japanese people who had extended 1 of my methods in a new direction.

And so I spent the next 5 days writing a program to implement what they did.

And then I, you know, they had only generalized part of what I had done.

So then I had to see if I could generalize more parts of it.

And then I had to take their approach, and I had to try it out on a couple of dozen of the other problems I had already worked out with my old methods.

And so that took another couple of weeks.

And then I started to see the light, and I started writing the final draft.

And then I would type it up, involve some new mathematical questions.

And so I wrote to my friends who might be good at solving those problems, and they solved some of them.

And so a month later, I had absorbed 1 new idea that I learned and I'm glad I heard about it in time.

Otherwise, I wouldn't put my book out before I'd heard about the idea.

On the other hand, this book was supposed to come in at 300 pages, and I'm up to 350 now.

But if I learn about another 1, my publisher's gonna shoot me.

Well, so in the process, in that 1 month process, are some days harder than others?

My work is fun, but I also work hard, and every big job has parts that are a lot more fun than others.

And so, many days I'll say, why do I have to have such high standards?

Why couldn't I just be sloppy and not try this out and just report the answer?

But I know that people are counting me to do this, And so, okay, so okay, Don, I'll grit my teeth and do it." And then the joy comes out when I see that actually I'm getting good results.

And even more when I see that somebody has actually read and understood what I wrote and told me how to make it even better.

I did want to mention something about the method.

So I got this tablet here where I do the first writing of concepts.

But here, you know, I'm saying, explain how to draw such skewed pixel diagrams.

got this paper about 40 years ago when I was visiting my sister in Canada and they make tablets of paper with this nice large size and just the

You know, I've got these manuscripts going back to the 60s.

And those are when I get my ideas on paper, okay?

In fact, I went to typing school when I was in high school, and so I can type faster than I think.

So then when I do the editing, stand up and type, then I revise this and it comes out a lot different than what, for style and rhythm and things like that come out at the typing state.

To a certain extent, I have only a small number of idioms that I use, like, you know, I'm beginning a theorem, I do something for, displayed equation, I do something and so on.

So for example, Turing wrote, what, the other direction.

You don't write macros, you don't think in macros.

Not particularly, but when I need a macro, I'll go ahead and do it.

I mean, I'll change something if I can save a line.

I'll figure out a way to rewrite the sentence so that it'll look better on the page.

And I shouldn't be wasting my time on that, but I can't resist because I know It's only another 3% of the time or something like that.

And it could also be argued that that is what life is about.

Like I work in the garden 1 day a week and that's kind of a description of my life is getting rid of weeds, removing bugs from programs.

So you know a lot of writers talk about basically suffering, the writing processes, having, It's extremely difficult.

And I think of programming, especially the technical writing that you're doing, can be like that.

Do you find yourself, methodologically, how do you, every day, sit down to do the work?

You kind of say it's, you know, it's fun.

But it'd be interesting to hear if there are non-fun parts that you really struggle with.

Yeah, so the fun comes when I'm able to put together ideas of 2 people who didn't know about each other.

And so I might be the first person that saw both of their ideas.

And so then I get to make the synthesis, and that gives me a chance to be creative.

But the dredge work is where I've got to chase everything down to its root.

This leads me into really interesting stuff.

I learn about Sanskrit, and I try to give credit to all the authors and so I write to people who know the people, authors if they're dead or I communicate this way.

And I gotta get the math right and I gotta tack all my programs, try to find holes in them.

And I rewrite the programs after I get a better idea.

1 of the things that I spent a lot of time preparing, a major example based on the game of baseball.

And I know a lot of people for whom baseball is the most important thing in the world.

But I also know a lot of people for whom cricket is the most important in the world or soccer or something.

And I realized that if I had a big example, I mean, it was going to have a fold-out illustration and everything.

And I was saying, well, what am I really teaching about algorithms here where I had this baseball example and if I was a person who knew only cricket, what would they think about this?

So, I've ripped the whole thing out, but I had something that would have really appealed to people who grew up with baseball as a major theme in their life.


Speaker 0: So what's your writing process like? What's your thinking and writing process like every day? What's your routine even?

Speaker 1: Yeah, I guess it's actually the best question because I spend 7 days a week doing

Speaker 0: it. You're the most prepared to answer it.

Speaker 1: Yeah, but okay, so the chair I'm sitting in is where I do...

Speaker 0: It's where the magic happens.

Speaker 1: Well, reading and writing. My chair is usually sitting over there where I have other books, some reference books. But I found this chair which was designed by a Swedish guy anyway. It turns out this is the only chair I can really sit in for hours and hours and not know that I'm in a chair But then I have the stand-up desk right next next to us and and so after I write something with pencil and eraser I get up and I type it and revise and rewrite. I'm standing up.

Speaker 0: The kernel of the idea is first put on paper.

Speaker 1: Yeah. That's where. Right, and I'll write maybe 5 programs a week, of course, literate programming. And these are, Before I describe something in my book, I always program it to see how it's working, and I try it a lot. For example, at the end of January, I learned of a breakthrough by 4 Japanese people who had extended 1 of my methods in a new direction.

And so I spent the next 5 days writing a program to implement what they did. And then I, you know, they had only generalized part of what I had done. So then I had to see if I could generalize more parts of it. And then I had to take their approach, and I had to try it out on a couple of dozen of the other problems I had already worked out with my old methods. And so that took another couple of weeks.

And then I started to see the light, and I started writing the final draft. And then I would type it up, involve some new mathematical questions. And so I wrote to my friends who might be good at solving those problems, and they solved some of them. So I put that in as exercises. And so a month later, I had absorbed 1 new idea that I learned and I'm glad I heard about it in time.

Otherwise, I wouldn't put my book out before I'd heard about the idea. On the other hand, this book was supposed to come in at 300 pages, and I'm up to 350 now. That added 10 pages to the book. But if I learn about another 1, my publisher's gonna shoot me. And...

Speaker 0: -'Cause... Well, so in the process, in that 1 month process, are some days harder than others?

Speaker 1: Are some days harder than others? Well, yeah. My work is fun, but I also work hard, and every big job has parts that are a lot more fun than others. And so, many days I'll say, why do I have to have such high standards? Why couldn't I just be sloppy and not try this out and just report the answer?

But I know that people are counting me to do this, And so, okay, so okay, Don, I'll grit my teeth and do it." And then the joy comes out when I see that actually I'm getting good results. And even more when I see that somebody has actually read and understood what I wrote and told me how to make it even better. I did want to mention something about the method. So I got this tablet here where I do the first writing of concepts.

Speaker 0: And what language is that then?

Speaker 1: Right. So, take a look at it. But here, you know, I'm saying, explain how to draw such skewed pixel diagrams. Okay, so

Speaker 0: I

Speaker 1: got this paper about 40 years ago when I was visiting my sister in Canada and they make tablets of paper with this nice large size and just the

Speaker 0: right size. And a very small space between lines.

Speaker 1: Small spaces, yeah, yeah.

Speaker 0: Maybe also just show it.

Speaker 1: Yeah.

Speaker 0: Yeah. Wow.

Speaker 1: You know, I've got these manuscripts going back to the 60s. And those are when I get my ideas on paper, okay? But I'm a good typist. In fact, I went to typing school when I was in high school, and so I can type faster than I think. So then when I do the editing, stand up and type, then I revise this and it comes out a lot different than what, for style and rhythm and things like that come out at the typing state.

Speaker 0: And you type in tech.

Speaker 1: And I type in tech.

Speaker 0: And can you think in tech?

Speaker 1: No. So. To a certain extent, I have only a small number of idioms that I use, like, you know, I'm beginning a theorem, I do something for, displayed equation, I do something and so on. But I have to see it.

Speaker 0: And the way that it's on paper here.

Speaker 1: Yeah, right.

Speaker 0: So for example, Turing wrote, what, the other direction. You don't write macros, you don't think in macros.

Speaker 1: Not particularly, but when I need a macro, I'll go ahead and do it. But the thing is, I also write to fit. I mean, I'll change something if I can save a line. It's like haiku. I'll figure out a way to rewrite the sentence so that it'll look better on the page.

And I shouldn't be wasting my time on that, but I can't resist because I know It's only another 3% of the time or something like that.

Speaker 0: And it could also be argued that that is what life is about.

Speaker 1: Ah, yes, in fact, that's true. Like I work in the garden 1 day a week and that's kind of a description of my life is getting rid of weeds, removing bugs from programs.

Speaker 0: So you know a lot of writers talk about basically suffering, the writing processes, having, It's extremely difficult. And I think of programming, especially the technical writing that you're doing, can be like that. Do you find yourself, methodologically, how do you, every day, sit down to do the work? Is it a challenge? You kind of say it's, you know, it's fun.

But it'd be interesting to hear if there are non-fun parts that you really struggle with.

Speaker 1: Yeah, so the fun comes when I'm able to put together ideas of 2 people who didn't know about each other. And so I might be the first person that saw both of their ideas. And so then I get to make the synthesis, and that gives me a chance to be creative. But the dredge work is where I've got to chase everything down to its root. This leads me into really interesting stuff.

I learn about Sanskrit, and I try to give credit to all the authors and so I write to people who know the people, authors if they're dead or I communicate this way. And I gotta get the math right and I gotta tack all my programs, try to find holes in them. And I rewrite the programs after I get a better idea.

Speaker 0: Is there ever dead ends?

Speaker 1: Dead ends? Oh, yeah. I throw stuff out, yeah. 1 of the things that I spent a lot of time preparing, a major example based on the game of baseball. And I know a lot of people for whom baseball is the most important thing in the world.

But I also know a lot of people for whom cricket is the most important in the world or soccer or something. And I realized that if I had a big example, I mean, it was going to have a fold-out illustration and everything. And I was saying, well, what am I really teaching about algorithms here where I had this baseball example and if I was a person who knew only cricket, what would they think about this? So, I've ripped the whole thing out, but I had something that would have really appealed to people who grew up with baseball as a major theme in their life.

Speaker 0: Which is a lot of people,

Speaker 1: but still

Speaker 0: a minority.

Speaker 1: Small minority, I took out bowling too.

Speaker 0: Even a smaller minority.

See all Lex Fridman transcripts on Youtube

Donald Knuth: Writing Process | AI Podcast Clips