George Hotz is a programmer, hacker, and the founder of comma-ai and tiny corp. Please support this podcast by checking out our sponsors:

- Numerai: https://numer.ai/lex

- Babbel: https://babbel.com/lexpod and use code Lexpod to get 55% off

- NetSuite: http://netsuite.com/lex to get free product tour

- InsideTracker: https://insidetracker.com/lex to get 20% off

- AG1: https://drinkag1.com/lex to get 1 year of Vitamin D and 5 free travel packs



Transcript: https://lexfridman.com/george-hotz-3-transcript



EPISODE LINKS:

George's Twitter: https://twitter.com/realgeorgehotz

George's Twitch: https://twitch.tv/georgehotz

George's Instagram: https://instagram.com/georgehotz

Tiny Corp's Twitter: https://twitter.com/__tinygrad__

Tiny Corp's Website: https://tinygrad.org/

Comma-ai's Twitter: https://twitter.com/comma_ai

Comma-ai's Website: https://comma.ai/

Comma-ai's YouTube (unofficial): https://youtube.com/georgehotzarchive

Mentioned:

Learning a Driving Simulator (paper): https://bit.ly/42T6lAN



PODCAST INFO:

Podcast website: https://lexfridman.com/podcast

Apple Podcasts: https://apple.co/2lwqZIr

Spotify: https://spoti.fi/2nEwCF8

RSS: https://lexfridman.com/feed/podcast/

YouTube Full Episodes: https://youtube.com/lexfridman

YouTube Clips: https://youtube.com/lexclips



SUPPORT & CONNECT:

- Check out the sponsors above, it's the best way to support this podcast

- Support on Patreon: https://www.patreon.com/lexfridman

- Twitter: https://twitter.com/lexfridman

- Instagram: https://www.instagram.com/lexfridman

- LinkedIn: https://www.linkedin.com/in/lexfridman

- Facebook: https://www.facebook.com/lexfridman

- Medium: https://medium.com/@lexfridman



OUTLINE:

Here's the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.

(00:00) - Introduction

(08:04) - Time is an illusion

(17:44) - Memes

(20:20) - Eliezer Yudkowsky

(32:45) - Virtual reality

(39:04) - AI friends

(46:29) - tiny corp

(59:50) - NVIDIA vs AMD

(1:02:47) - tinybox

(1:14:56) - Self-driving

(1:29:35) - Programming

(1:37:31) - AI safety

(2:02:29) - Working at Twitter

(2:40:12) - Prompt engineering

(2:46:08) - Video games

(3:02:23) - Andrej Karpathy

(3:12:28) - Meaning of life

The following is a conversation with George Hautz.

He's the founder of kama AI that seeks to solve a time was driving and is the founder of a new company called Tiny Corp.

That created Tiny Grad, a neural network framework that is extremely simple.

With the goal of making it run on any device by any human, easily and officially.

As you know, George also did a large number of fun and amazing things from hacking the iPhone to recently joining Twitter for a bit as a in turn in quotes, making the case for refactoring the Twitter code base.

In general, he's a fascinating engineer and human being and 1 of my favorite people to talk to.

And now a quick few second mention of e sponsor.

Check them out in the description is the best way to support this podcast.

Who got Numerai for the world's hardest data science tournament, Babbel for learning new languages, that suite for business management software insight tracker for blood paneling and AG1 for my daily multi vitamin.

Also, If you want to work on our team, we're always hiring.

Go to next freemium dot com slash hiring.

I try to make this interesting, but if you must skip them, friends, please still check out our sponsors.

This episode is brought to you by Numereye, a hedge fund that uses artificial intelligence and machine learning to make investment decisions They created a tournament that challenges data scientists to build best predictive models for financial markets.

It's basically just a really, really difficult real world data set to test out your ideas for how to build machine learning models.

I think this is a great educational platform.

I think a great way to explore, to learn about machine learning, to really test yourself on real world data with consequences.

The models are scored based on how well they perform on unseen data, and the top performers receive a share of the turbines prize pool.

Head over to numeral I slash lex, that's NUMER dot a I slash lex to sign up for a tournament and hone your machine learning skills.

That's numerous I slash lex for a chance to play against me and win the share of the Tournament's prize pool.

This show is also brought to you by Babbel, an app and website that gets you speaking in a new language within weeks I have been using it to learn a few languages, Spanish, to review Russian, to practice Russian, to revisit Russian from a different perspective because that becomes more and more relevant for some of the previous conversations I've had and some upcoming conversations I have.

It really is fascinating how much another language, knowing another language, even to a degree where you could just have little bits and pieces of a conversation can really unlock an experience in another part of the world.

When you travel in France, in Paris, just having a few words at your disposal, a few phrases.

It begins to really open you up to strange, fascinating new experiences that ultimately, at least to me, teach me that we're all the same.

We have to first see our differences to realize those differences are grounded in the basic humanity.

And that experience that were all very different and yet at the core of the same.

I think travel with the aid of language really helps unlock.

You can get 55 percent off your babble subscription at babble dot com slash lax pod that's spelled BABBEL dot com slash lex pod.

This shows also brought to you by NetSuite.

An all in 1 cloud business management system that manage all the messy stuff that is required to run a business, the financials, the human resources, the inventory.

If you do that kind of thing, e commerce, all that stuff, all the business related details.

I know how stressed I am about everything that's required to run a team.

To run a business that involves much more than just ideas and designs and engineering.

And evolves all the management of human beings, all the complexities of that, the financials, all of it, and so you should be using the best tools for the job.

I sometimes wonder if I have it in me mentally and skill wise to be a part of running a large company.

I think, like, with a lot of things in life, it's 1 of those things you shouldn't wonder too much about.

But again, using the best tools for the job, is required here.

You can start now with a no payment or interest for 6 months, go to netsuite dot com slash to access their 1 of a kind financing program, that's net suite dot com slash flex.

This show was also brought to buy Inside Tracker, a service I used to track biological data, data that comes from my body, to predict.

To tell me what I should do with my lifestyle, my diet, what's working, and what's not working.

It's obvious all the exciting breakthroughs that are happening with transformers, with large language models, even with diffusion, all of that is obvious that with raw data, with huge amounts of raw data, fine tuned to the individual, would really reveal to us the signal in all the noise of biology.

The kinds of leaps in development that we saw in language and now more and more visual data.

Feel like biological data is around the corner.

In this multi hierarchical distributed system that is our biology.

What is The thing that it's missing that could be aided.

Simple changes in all kinds of things that are controllable by individual human being.

I can't wait till that's a possibility, and Insight Tracker is taking steps towards that.

Because special savings for a limited time when you go to insight tracker dot com slash flex.

This show is also brought to you by Athleta Greens, that's now called AG1.

At the very least, it's an all in 1 daily drink to sport better health and peak performance.

It helps me reconnect with the basics, the nutritional basics that makes this whole machine that is our human body run.

All the crazy mental stuff I do for work, the physical challenges, everything.

All of that is somehow made better knowing that at least you got nutrition in check.

At least you're getting enough sleep, at least you're doing the basics, at least you're doing the exercise.

Once you get those basics in place, I think you can do some quite difficult things in life.

But anyway, beyond all that, it's just a source of happiness and and a kind of a feeling of home, the feeling that comes from returning to the habit time and time again.

Anyway, they'll give you 1 month supply of fish oil when you sign up at drink AG1 dot com slash flex.

This is Alex Frutman podcast to support it.

Please check out our sponsors in the description.

And now to your friends, here's George Hottz.

You mentioned something in a stream about the philosophical nature of time.

You know, I sell phone calls to comma for a thousand dollars.

And some guy called me and, like, you know, it's a thousand dollars.

And I really wanted to share this with you.

I'm like, oh, what do you mean time doesn't exist?

It's about whether it's a useful model to describe reality?

Do you think there is an objective reality or is everything just useful models?

Like underneath it all, is it an actual thing that we're constructing models for?

I mean, this kind of connects to the models with constructive reality with machine learning.

Like, is it just nice to have useful approximations of the world such that we can do something with it?

And I think hard things are actually hard.

I don't think p equals NP0, strong words.

I don't think you're the person that follows the majority and all walks of life.

In theoretical computer science, you you're you're 1 of the

What were you talking about on the stream with time?

If I remembered half the things I said on stream.

Someday someone's gonna make a model of all of that.

Would that be exciting to you or said that there's a George Hott's model

I mean, the question is when the George Hotz model is better than George Hotz.

Like, I am declining and the model is growing.

the metric by which you measure better or in that if you're competing with yourself.

Maybe you can just play a game where you have the George Hotz answer and the George Hotz model answer and ask which people prefer.

Either 1, it will hurt more when it's people close to me, but both will be overtaken by the George Hott's model.

Family members would rather have the model over for Thanksgiving than you.

Or like significant others would rather sexed.

With the with the large language model version of you,

especially when it's fine tuned to their preferences.

Well, that's what we're doing in a relationship.

We're just fine tuning ourselves, but we're inefficient with it because we're selfish and greedy and so on.

Language models can fine tune more efficiently, more selflessly.

There's a star check for Voyager episode where, you know, Catherine Drainway lost in the Delta quadrant makes herself a lover on the holidays.

And the lover falls asleep on her arm and he snores a little bit and, you know, Jay might add it to the program to remove that.

And then, of course, the realization is, wait, this person's terrible.

It is actually all their nuances and quirks and slight annoyances that that make this relationship worthwhile.

But I don't think we're gonna realize that until it's too late.

Well, I think a large language model could incorporate the flaws -- Uh-huh.

-- and the quirks and all that kind of stuff.

Just the perfect amount of quirks and efflower flaws to make you charming without crossing the line.

And that's probably a good, like, approximation of the, like, the percent of time the language model should be cranky or an asshole -- Yeah.

But all that difficulty at that point is artificial.

What's the difference between real and artificial?

Artificial difficulty is a difficulty that's, like, constructed or could be turned off with a knob.

Real difficulty is, like, you're in the woods and you gotta survive.

if something cannot be turned off with a knob is real.

Or, I mean, you can't get out of this by smashing the knob with a hammer.

I mean, maybe you kind of can, you know, into the wild when you know, Alexander Supertramp.

He wants to explore something that's never been explored before, but it's the nineties.

So he's like, well, I'm just not gonna bring a map.

There was a bridge a mile from where you were camping.

How does it connect to the metaphor of the knob?

By not bringing the map, you didn't become an explorer.

The art the difficulty is still artificial.

What if we just don't have access to the knob?

Like, we already exist in a world of nature.

And nature has been fine tuned over billions of years.

To have humans build something and then throw the knob away and some grand romantic gestures horrifying?

Do you think of us humans as individuals that are like born and die?

Or is it are we just all part of 1 living organism?

I don't think there's a clear line there.

Everything running your head is just the it's not it's this computation.

Everything running in the universe is computation, I think.

I I believe the extended church starting thesis.

But it there seems to be an embodiment to your particular computation.

But, I mean, models have consistency too.

Models that have been RLHF will continually say, you know, like, well, how do I murder ethnic minorities.

We we find we provide human feedback and then there that thereby fine tune these little pockets of computation.

But it's still unclear why that pocket of computation stays with you, like, for years.

It's just kind of thought, like, you have this consistent set of physics biology, what like, whatever you call the the neurons firing.

Like like the electrical signals, the mechanical signals, all of that, that seems to stay there, and it contains information, stores information, and that information permeates through time and stays with you.

To be fair, like a lot of the models we're building today are very even RLHF is nowhere near as complex as the human loss function.

Reinforcement learning with human feedback.

You know, when I talked about, will GPT 12 be AGI?

I mean, cross entry loss is never gonna get you there.

You need probably RL in fancy environments in order to get something that would be considered like AGI like.

So to ask, like, the question about, like, why?

I don't think there's anything particularly special about Where I ended up?

Look, actually, I I don't really even like the word AGI, but general intelligence is defined to be whatever humans have.

If your loss function is categorical cross entropy, if your loss function is just try to maximize compression, I have a soundcloud I wrap.

And I tried to get chat UPT to help me write wraps.

And the wraps that it wrote sounded like YouTube common wraps.

You know, you can go on any wrap sheet online and you can see what people put in the comments.

And it's the most, like, mid quality wrap you can find.

Every time I talk to you, I learn new words.

So there's like there's like a, like, see that intelligence curve.

Where and you have, like, the dumb guy, the smart guy, and in the mid guy.

Actually, being in the mid guys the worst.

The smart guy was like, I put all my money in Bitcoin.

The mid guy is like, you can't put money in Bitcoin.

See all Lex Fridman Podcast transcripts on Applepodcasts

#387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God