The focus should be on how much time is spent rather than what to do. Spending 10,000 hours is important to become an expert.

Advice for Beginners in Machine Learning

Forming a daily habit and comparing progress to oneself from the past can help in achieving the goal of 10,000 hours.

Maximizing Likelihood of Getting to 10,000 Hours

Making mistakes and accumulating scar tissue can lead to growth and learning. Focus on the work done rather than worrying about the right path to take.

Overcoming Paralysis by Choice

Teaching is not the most enjoyable activity, but being helpful to others and seeing happy humans appreciating the teaching makes it worthwhile.

Teaching and Happy Humans

Creating good educational materials requires a lot of iteration, thought, and effort. It can be challenging but rewarding.

Creating Educational Content

Teaching helps strengthen understanding and reveals gaps in knowledge. Explaining concepts to others is a way to learn.

Learning through Teaching

Using code as the source of truth in teaching provides a practical and tangible learning experience.

The Power of Building and Running Code

## Newsletter Content:

- You're 1 of the greatest teachers of machine learning, AI ever, from CS231N to today.
- Advice for beginners interested in getting into machine learning:
  - Focus on the quantity of hours rather than what to do.
  - Aim to spend 10,000 hours of deliberate effort and work.
  - It doesn't matter where you put in the hours, as long as you put them in.
  - Iteration and improvement come with time and practice.
- Maximizing your likelihood of reaching 10,000 hours:
  - Form a daily habit of working on machine learning.
  - Focus on the psychological aspect and overcome self-comparisons.
  - Compare yourself to your past self and track progress.
- Overcoming paralysis by choice:
  - Don't worry about making the wrong choices.
  - Mistakes and scar tissue contribute to growth and learning.
- Teaching as a way to learn and strengthen understanding.
- The process of creating educational content:
  - Multiple takes and selecting the best ones.
  - Iteration and refinement.
  - Approximately 10 hours to create 1 hour of content.
- Going back to basics and gaining wisdom from it.
- The power of coding and building to understand concepts.

Continue reading the transcript for more insights into machine learning and teaching.

The Path to Becoming an Expert in Machine Learning

Focus on the quantity of hours spent on learning.

It guarantees success in machine learning.

Comparing yourself to others in the field.

Avoiding wasting time on wrong approaches.

Comparing themselves to others in the field.

Mistakes help in learning from past experiences.

The ease of creating educational materials.

The amount of iteration and thought required.

The difficulty of finding relevant content.

The time-consuming process of preparation.

Using mathematical symbols and equations.

- The speaker advises beginners in machine learning to focus on the amount of time they spend learning rather than what specific tasks they should do.
- They suggest putting in 10,000 hours of deliberate effort and work to become an expert in any field.
- Forming a daily habit of practicing is important for maximizing progress.
- The speaker encourages comparing oneself to their past self rather than others.
- Making mistakes and learning from them is valuable in the learning process.
- The speaker mentions that teaching is not always enjoyable, but they find satisfaction in helping others and strengthening their own understanding.
- Creating educational content takes a lot of time and iteration.
- The speaker's teaching style involves building and coding in real-time to provide a practical understanding of concepts.

Lex Fridman Podcast full episode: https://www.youtube.com/watch?v=cdiD-9MMpb0
Please support this podcast by checking out our sponsors:
- Eight Sleep: https://www.eightsleep.com/lex to get special savings
- BetterHelp: https://betterhelp.com/lex to get 10% off
- Fundrise: https://fundrise.com/lex
- Athletic Greens: https://athleticgreens.com/lex to get 1 month of fish oil

GUEST BIO:
Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

SOCIAL:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Reddit: https://reddit.com/r/lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

You're 1 of the greatest teachers of machine learning, AI ever, from CS231N to today.

What advice would you give to beginners interested in getting into machine learning?

Beginners are often focused on like what to do.

And I think the focus should be more like how much you do.

So I am kind of like believer on a high level in this 10,000 hours kind of concept where you just kind of have to just pick the things where you can spend time and you care about and you're interested in, you literally have to put in 10,000 hours of work.

It doesn't even matter as much where you put it.

And you'll iterate and you'll improve and you'll waste some time.

But I think it's actually really nice because I feel like there's some sense of determinism about being an expert at a thing if you spend 10,000 hours.

You can literally pick an arbitrary thing, and I think if you spend 10,000 hours of deliberate effort and work, you actually will become an expert at it.

And so I think that's kind of like a nice thought.

And so basically I would focus more on like, are you spending 10,000 hours?

So, and then thinking about what kind of mechanisms maximize your likelihood of getting to 10,000

Which for us silly humans means probably forming a daily habit of like every single day actually doing the thing.

So I do think to a large extent it's a psychological problem for yourself.

1 other thing that I think is helpful for the psychology of it, is many times people compare themselves to others in the area.

Only compare yourself to you from some time ago, like say a year ago.

And I think this, then you can see your progress and it's very motivating.

That's so interesting that focus on the quantity of hours.

Because I think a lot of people in the beginner stage, but actually throughout, get paralyzed by the choice, like which 1, do I pick this path or this path?

Like they'll literally get paralyzed by like which IDE to use.

Yeah, they're worried about all these things.

But the thing is, some of the, you will waste time doing something wrong.

You will eventually figure out it's not right.

And next time you'll grow stronger because next time you'll have the scar tissue and next time you'll learn from it.

And now next time you come to a similar situation, you'll be like, oh, I messed up.

I've spent a lot of time working on things that never materialize into anything.

And I have all that scar tissue, and I have some intuitions about what was useful, what wasn't useful, how things turned out.

So all those mistakes were not dead work.

So I just think you should, did you just focus on working?

That's a good question actually to ask for a lot of things, not just machine learning.

It's a good way to cut the, I forgot what the term we used, but the fluff, the blubber, whatever the, the inefficiencies in life.

You seem to find yourself often in the, like drawn to teaching.

You're very good at it, but you're also drawn to it.

I mean, I don't think I love teaching, I love happy humans.

I wouldn't say I hate teaching, I tolerate teaching, but it's not like the act of teaching that I like, it's that I have something, I'm actually okay at it, I'm okay at teaching, and people appreciate it a lot.

And so I'm just happy to try to be helpful.

And teaching itself is not the most, I mean, it can be really annoying, frustrating.

I was working on a bunch of lectures just now.

I was reminded back to my days of 231N just how much work it is to create some of these materials and make them good.

The amount of iteration and thought, and you go down blind alleys and just how much you change it.

So creating something good in terms of like educational value is really hard.

So people should definitely go watch your new stuff.

You put out, there are lectures where you're actually building the thing, like from, like you said, the code is truth.

So discussing back propagation by building it, by looking through it, just the whole thing.

I think that's a really powerful way to teach.

Did you have to prepare for that or are you just live thinking through it?

I will typically do like say 3 takes and then I take like the better take.

So I do multiple takes and I take some of the better takes and then I just build out a lecture that way.

Sometimes I have to delete 30 minutes of content because it just went down an alley that I didn't like too much.

So there's a bunch of iteration and it probably takes me, you know, somewhere around 10 hours to create 1 hour

I mean, is it difficult to go back to the basics?

Do you draw a lot of like wisdom from going back to the basics?

Yeah, going back to backpropagation loss functions, where they come from.

And 1 thing I like about teaching a lot, honestly, is it definitely strengthens your understanding.

So it's not a purely altruistic activity.

If you have to explain something to someone, you realize you have gaps in knowledge.

And so I even surprised myself in those lectures.

Like, oh, so the result will obviously look at this and then the result doesn't look like it.

And I'm like, okay, I thought I understood this.

They literally code, you run it in a notebook and it gives you a result and you're like, oh, Wow.

And like actual numbers, actual input, actual code.

Yeah, it's not mathematical symbols, etc.


Speaker 0: You're 1 of the greatest teachers of machine learning, AI ever, from CS231N to today. What advice would you give to beginners interested in getting into machine learning?

Speaker 1: Beginners are often focused on like what to do. And I think the focus should be more like how much you do. So I am kind of like believer on a high level in this 10,000 hours kind of concept where you just kind of have to just pick the things where you can spend time and you care about and you're interested in, you literally have to put in 10,000 hours of work. It doesn't even matter as much where you put it. And you'll iterate and you'll improve and you'll waste some time.

I don't know if there's a better way. You need to put in 10,000 hours. But I think it's actually really nice because I feel like there's some sense of determinism about being an expert at a thing if you spend 10,000 hours. You can literally pick an arbitrary thing, and I think if you spend 10,000 hours of deliberate effort and work, you actually will become an expert at it. And so I think that's kind of like a nice thought.

And so basically I would focus more on like, are you spending 10,000 hours?

Speaker 0: That's what I'd focus on. So, and then thinking about what kind of mechanisms maximize your likelihood of getting to 10,000

Speaker 1: hours. Exactly.

Speaker 0: Which for us silly humans means probably forming a daily habit of like every single day actually doing the thing.

Speaker 1: Whatever helps you. So I do think to a large extent it's a psychological problem for yourself. 1 other thing that I think is helpful for the psychology of it, is many times people compare themselves to others in the area. I think this is very harmful. Only compare yourself to you from some time ago, like say a year ago.

Are you better than you a year ago? It's the only way to think. And I think this, then you can see your progress and it's very motivating.

Speaker 0: That's so interesting that focus on the quantity of hours. Because I think a lot of people in the beginner stage, but actually throughout, get paralyzed by the choice, like which 1, do I pick this path or this path? Like they'll literally get paralyzed by like which IDE to use. Well, they're worried.

Speaker 1: Yeah, they're worried about all these things. But the thing is, some of the, you will waste time doing something wrong. You will eventually figure out it's not right. You will accumulate scar tissue. And next time you'll grow stronger because next time you'll have the scar tissue and next time you'll learn from it.

And now next time you come to a similar situation, you'll be like, oh, I messed up. I've spent a lot of time working on things that never materialize into anything. And I have all that scar tissue, and I have some intuitions about what was useful, what wasn't useful, how things turned out. So all those mistakes were not dead work. So I just think you should, did you just focus on working?

What have you done? What have you done last week?

Speaker 0: That's a good question actually to ask for a lot of things, not just machine learning. It's a good way to cut the, I forgot what the term we used, but the fluff, the blubber, whatever the, the inefficiencies in life. What do you love about teaching? You seem to find yourself often in the, like drawn to teaching. You're very good at it, but you're also drawn to it.

Speaker 1: I mean, I don't think I love teaching, I love happy humans. And happy humans like when I teach. I wouldn't say I hate teaching, I tolerate teaching, but it's not like the act of teaching that I like, it's that I have something, I'm actually okay at it, I'm okay at teaching, and people appreciate it a lot. And so I'm just happy to try to be helpful. And teaching itself is not the most, I mean, it can be really annoying, frustrating.

I was working on a bunch of lectures just now. I was reminded back to my days of 231N just how much work it is to create some of these materials and make them good. The amount of iteration and thought, and you go down blind alleys and just how much you change it. So creating something good in terms of like educational value is really hard. And it's not fun.

Speaker 0: It was difficult. So people should definitely go watch your new stuff. You put out, there are lectures where you're actually building the thing, like from, like you said, the code is truth. So discussing back propagation by building it, by looking through it, just the whole thing. So how difficult is that to prepare for?

I think that's a really powerful way to teach. Did you have to prepare for that or are you just live thinking through it?

Speaker 1: I will typically do like say 3 takes and then I take like the better take. So I do multiple takes and I take some of the better takes and then I just build out a lecture that way. Sometimes I have to delete 30 minutes of content because it just went down an alley that I didn't like too much. So there's a bunch of iteration and it probably takes me, you know, somewhere around 10 hours to create 1 hour

Speaker 0: of content. To get 1 hour. It's interesting. I mean, is it difficult to go back to the basics? Do you draw a lot of like wisdom from going back to the basics?

Speaker 1: Yeah, going back to backpropagation loss functions, where they come from. And 1 thing I like about teaching a lot, honestly, is it definitely strengthens your understanding. So it's not a purely altruistic activity. It's a way to learn. If you have to explain something to someone, you realize you have gaps in knowledge.

And so I even surprised myself in those lectures. Like, oh, so the result will obviously look at this and then the result doesn't look like it. And I'm like, okay, I thought I understood this. Yeah. Yeah.

Speaker 0: Well, that's why it's really cool. They literally code, you run it in a notebook and it gives you a result and you're like, oh, Wow. Yes. And like actual numbers, actual input, actual code.

Speaker 1: Yeah, it's not mathematical symbols, etc. The source of truth is the code. It's not slides. It's just like, let's build it.

Speaker 0: It's beautiful. You're a rare human in that sense. You you

See all Lex Clips transcripts on Youtube

Advice for machine learning beginners | Andrej Karpathy and Lex Fridman