See all Lex Fridman Podcast transcripts on Applepodcasts

The following is a conversation with George Hautz. His third time in his podcast. He's the founder of kama AI that seeks to solve a time was driving and is the founder of a new company called Tiny Corp. That created Tiny Grad, a neural network framework that is extremely simple. With the goal of making it run on any device by any human, easily and officially.

As you know, George also did a large number of fun and amazing things from hacking the iPhone to recently joining Twitter for a bit as a in turn in quotes, making the case for refactoring the Twitter code base. In general, he's a fascinating engineer and human being and 1 of my favorite people to talk to. And now a quick few second mention of e sponsor. Check them out in the description is the best way to support this podcast. Who got Numerai for the world's hardest data science tournament, Babbel for learning new languages, that suite for business management software insight tracker for blood paneling and AG1 for my daily multi vitamin.

Choose wisely my friends. Also, If you want to work on our team, we're always hiring. Go to next freemium dot com slash hiring. And now I want to the full ad reads. As always, no ads in the middle.

I try to make this interesting, but if you must skip them, friends, please still check out our sponsors. I enjoy their stuff. Maybe you will too. This episode is brought to you by Numereye, a hedge fund that uses artificial intelligence and machine learning to make investment decisions They created a tournament that challenges data scientists to build best predictive models for financial markets. It's basically just a really, really difficult real world data set to test out your ideas for how to build machine learning models.

I think this is a great educational platform. I think a great way to explore, to learn about machine learning, to really test yourself on real world data with consequences. No financial background is needed. The models are scored based on how well they perform on unseen data, and the top performers receive a share of the turbines prize pool. Head over to numeral I slash lex, that's NUMER dot a I slash lex to sign up for a tournament and hone your machine learning skills.

That's numerous I slash lex for a chance to play against me and win the share of the Tournament's prize pool. That's numeral I flex. This show is also brought to you by Babbel, an app and website that gets you speaking in a new language within weeks I have been using it to learn a few languages, Spanish, to review Russian, to practice Russian, to revisit Russian from a different perspective because that becomes more and more relevant for some of the previous conversations I've had and some upcoming conversations I have. It really is fascinating how much another language, knowing another language, even to a degree where you could just have little bits and pieces of a conversation can really unlock an experience in another part of the world. When you travel in France, in Paris, just having a few words at your disposal, a few phrases.

It begins to really open you up to strange, fascinating new experiences that ultimately, at least to me, teach me that we're all the same. We have to first see our differences to realize those differences are grounded in the basic humanity. And that experience that were all very different and yet at the core of the same. I think travel with the aid of language really helps unlock. You can get 55 percent off your babble subscription at babble dot com slash lax pod that's spelled BABBEL dot com slash lex pod.

Rules and restrictions apply. This shows also brought to you by NetSuite. An all in 1 cloud business management system that manage all the messy stuff that is required to run a business, the financials, the human resources, the inventory. If you do that kind of thing, e commerce, all that stuff, all the business related details. I know how stressed I am about everything that's required to run a team.

To run a business that involves much more than just ideas and designs and engineering. And evolves all the management of human beings, all the complexities of that, the financials, all of it, and so you should be using the best tools for the job. I sometimes wonder if I have it in me mentally and skill wise to be a part of running a large company. I think, like, with a lot of things in life, it's 1 of those things you shouldn't wonder too much about. You should either do or not do.

But again, using the best tools for the job, is required here. You can start now with a no payment or interest for 6 months, go to netsuite dot com slash to access their 1 of a kind financing program, that's net suite dot com slash flex. This show was also brought to buy Inside Tracker, a service I used to track biological data, data that comes from my body, to predict. To tell me what I should do with my lifestyle, my diet, what's working, and what's not working. It's obvious all the exciting breakthroughs that are happening with transformers, with large language models, even with diffusion, all of that is obvious that with raw data, with huge amounts of raw data, fine tuned to the individual, would really reveal to us the signal in all the noise of biology.

I feel like that's in the horizon. The kinds of leaps in development that we saw in language and now more and more visual data. Feel like biological data is around the corner. Unlocking what's there. In this multi hierarchical distributed system that is our biology.

What is it telling us? What is the sequence it holds? What is The thing that it's missing that could be aided. Simple lifestyle changes. Simple diet changes.

Simple changes in all kinds of things that are controllable by individual human being. I can't wait till that's a possibility, and Insight Tracker is taking steps towards that. Because special savings for a limited time when you go to insight tracker dot com slash flex. This show is also brought to you by Athleta Greens, that's now called AG1. It has the AG1 drink.

I drink it twice a day. At the very least, it's an all in 1 daily drink to sport better health and peak performance. I drink it cold. It's refreshing. It's grounding.

It helps me reconnect with the basics, the nutritional basics that makes this whole machine that is our human body run. All the crazy mental stuff I do for work, the physical challenges, everything. The highs and lows of life itself. All of that is somehow made better knowing that at least you got nutrition in check. At least you're getting enough sleep, at least you're doing the basics, at least you're doing the exercise.

Once you get those basics in place, I think you can do some quite difficult things in life. But anyway, beyond all that, it's just a source of happiness and and a kind of a feeling of home, the feeling that comes from returning to the habit time and time again. Anyway, they'll give you 1 month supply of fish oil when you sign up at drink AG1 dot com slash flex. This is Alex Frutman podcast to support it. Please check out our sponsors in the description.

And now to your friends, here's George Hottz. You mentioned something in a stream about the philosophical nature of time. So let's start with the wild question. Do you think time is an illusion?

You know, I sell phone calls to comma for a thousand dollars. And some guy called me and, like, you know, it's a thousand dollars. You can talk to me for for half an hour. And he's like, yeah. Okay.

So, like, time doesn't exist. And I really wanted to share this with you. I'm like, oh, what do you mean time doesn't exist? Right? Like, I think time is a useful model.

Whether it exists or not. Right? Like, does quantum physics exist? What doesn't matter? It's about whether it's a useful model to describe reality?

Is time maybe compressive.

Do you think there is an objective reality or is everything just useful models? Like underneath it all, is it an actual thing that we're constructing models for?

I don't know.

I was hoping you would know.

I don't think it matters.

I mean, this kind of connects to the models with constructive reality with machine learning. Right?

Sure.

Like, is it just nice to have useful approximations of the world such that we can do something with it?

So there are things that are real. Colm graph complexity is real.

Yeah.

Yeah. The compressive thing. Math is real.

Yeah. Should be a t shirt.

And I think hard things are actually hard. I don't think p equals NP0, strong words. Well, I think that's the majority. I do think Factoring is in Pete, but

I don't think you're the person that follows the majority and all walks of life. So but it's good

for that 1, I do.

Yeah. In theoretical computer science, you you're you're 1 of the

sheep. Alright.

But do you time is a useful model? Sure. What were you talking about on the stream with time? Are you made of time?

If I remembered half the things I said on stream. Someday someone's gonna make a model of all of that. I'm just gonna come back to haunt me

some day soon?

Yeah. Probably.

Would that be exciting to you or said that there's a George Hott's model

I mean, the question is when the George Hotz model is better than George Hotz. Like, I am declining and the model is growing. What is

the metric by which you measure better or in that if you're competing with yourself.

Maybe you can just play a game where you have the George Hotz answer and the George Hotz model answer and ask which people prefer. People close to you or strangers. Either 1, it will hurt more when it's people close to me, but both will be overtaken by the George Hott's model.

It'd be quite painful. Right? Loved ones. Family members would rather have the model over for Thanksgiving than you.

Yeah.

Or like significant others would rather sexed. With the with the large language model version of you,

especially when it's fine tuned to their preferences.

Yeah. Well, that's what we're doing in a relationship. Right? We're just fine tuning ourselves, but we're inefficient with it because we're selfish and greedy and so on. Language models can fine tune more efficiently, more selflessly.

There's a star check for Voyager episode where, you know, Catherine Drainway lost in the Delta quadrant makes herself a lover on the holidays.

Mhmm.

And the lover falls asleep on her arm and he snores a little bit and, you know, Jay might add it to the program to remove that. And then, of course, the realization is, wait, this person's terrible. It is actually all their nuances and quirks and slight annoyances that that make this relationship worthwhile. But I don't think we're gonna realize that until it's too late.

Well, I think a large language model could incorporate the flaws -- Uh-huh. -- and the quirks and all that kind of stuff.

Just the perfect amount of quirks and efflower flaws to make you charming without crossing the line.

Yeah. Yeah. And that's probably a good, like, approximation of the, like, the percent of time the language model should be cranky or an asshole -- Yeah. -- or jealous or all this kind of stuff.

And, of course, it can and it will. But all that difficulty at that point is artificial. There's no more real difficulty.

Okay? What's the difference between real and artificial?

Artificial difficulty is a difficulty that's, like, constructed or could be turned off with a knob. Real difficulty is, like, you're in the woods and you gotta survive. So

if something cannot be turned off with a knob is real.

Yeah. I think so. Or, I mean, you can't get out of this by smashing the knob with a hammer. I mean, maybe you kind of can, you know, into the wild when you know, Alexander Supertramp. He wants to explore something that's never been explored before, but it's the nineties.

Everything's been explored. So he's like, well, I'm just not gonna bring a map.

Yeah.

I mean, no. You're you're not exploring. You should've brought a map to you. You died. There was a bridge a mile from where you were camping.

How does it connect to the metaphor of the knob?

By not bringing the map, you didn't become an explorer. You just smashed the thing.

Yeah.

Yeah. The art the difficulty is still artificial.

You failed before you started. What if we just don't have access to the knob?

Well, that maybe is even scarier. Right? Like, we already exist in a world of nature. And nature has been fine tuned over billions of years. Yeah.

To have humans build something and then throw the knob away and some grand romantic gestures horrifying?

Do you think of us humans as individuals that are like born and die? Or is it are we just all part of 1 living organism? That is earth, that is nature.

I don't think there's a clear line there. I think it's all kinda just fuzzy. I don't know. I mean, I don't think I'm conscious. I don't think I'm anything.

I think I'm just a computer program.

So it's all computation. Everything running your head is just the it's not it's this computation.

Everything running in the universe is computation, I think. I I believe the extended church starting thesis.

Yeah. But it there seems to be an embodiment to your particular computation. Like, there's a consistency.

Well, yeah. But, I mean, models have consistency too. Yeah. Models that have been RLHF will continually say, you know, like, well, how do I murder ethnic minorities. Oh, well, I can't let you do that, Al.

There's a consistency to that behavior.

It's all RLHF. Like we are RLHF, each other. We we find we provide human feedback and then there that thereby fine tune these little pockets of computation. But it's still unclear why that pocket of computation stays with you, like, for years. It's just kind of thought, like, you have this consistent set of physics biology, what like, whatever you call the the neurons firing.

Like like the electrical signals, the mechanical signals, all of that, that seems to stay there, and it contains information, stores information, and that information permeates through time and stays with you. There's like memory. There's like sticky.

Okay. To be fair, like a lot of the models we're building today are very even RLHF is nowhere near as complex as the human loss function.

Reinforcement learning with human feedback.

You know, when I talked about, will GPT 12 be AGI? My answer is no. Of course not. I mean, cross entry loss is never gonna get you there. You need probably RL in fancy environments in order to get something that would be considered like AGI like.

So to ask, like, the question about, like, why? I don't know. Like, it's just some quirk of evolution. Right? I don't think there's anything particularly special about Where I ended up?

Where humans ended up?

So, okay. We have human level intelligence. Would you call the AGI? Whatever we have. G I.

Look, actually, I I don't really even like the word AGI, but general intelligence is defined to be whatever humans have.

Okay. So why can GPT 12 not get us to AGI? Can we just, like, linger on that?

If your loss function is categorical cross entropy, if your loss function is just try to maximize compression, I have a soundcloud I wrap. And I tried to get chat UPT to help me write wraps.

Mhmm.

And the wraps that it wrote sounded like YouTube common wraps. You know, you can go on any wrap sheet online and you can see what people put in the comments. And it's the most, like, mid quality wrap you can find.

Is me good or bad?

Mid is bad. Mid is bad. It's like mid. It's like

Every time I talk to you, I learn new words.

Mid. Yeah.

I was like I is it is it like basic? Is that what mid means?

I know. It's like it's like middle of the curve. Right? Yeah. So there's like there's like a, like, see that intelligence curve.

Yeah. Where and you have, like, the dumb guy, the smart guy, and in the mid guy. Actually, being in the mid guys the worst. The smart guy was like, I put all my money in Bitcoin. The mid guy is like, you can't put money in Bitcoin.

It's not real money.

And all of it is a genius meme. That's another interesting 1. MEMS. The humor, the idea, the absurdity encapsulated in a single image, and it just kind of propagates virally between all of our brains. I didn't get my sleep last night, so I'm very I sound like I'm high.

That's where I'm not. Do you think we have ideas or ideas have us?

I think that we're gonna get super scary memes once the AI's actually are superhuman.

Like every guy I would generate memes of course. Do you think it'll make humans laugh?

I think it's worse than that. So Infantagest, it's introduced in the first 50 pages, is about a tape that you once you watch it, once you only ever wanna watch that tape. In fact, you wanna watch the tape so much that someone says, okay, here's a hacksaw. Cut off your pinky. And then I'll let you watch the tape again and you'll do it.

So we're actually gonna build that I think, but it's not gonna be 1 static tape. I think the human brain is too complex. To be stuck in 1 static tape like that. If you look at like antbrands, maybe they can be stuck on a static tape. But we're going to build that using generative models.

Going to build the TikTok that you actually can't look away from.

So TikTok is already pretty close there, but the generation is done by humans. Yeah. The algorithm is just doing their recommendation. But if this do if the algorithm is also able to do the generation

well, it's a question about how much intelligence is behind it. Right? So the content is being generated by, let's say, 1 humanity worth of intelligence. And you can quantify a humanity. Right?

That's you know, it's it's exaflops, yada flops, but you can quantify it. Once that generation is being done by a hundred humanities, You're done.

So it's actually scale. That's the problem. But also speed. Yeah. And what if it's sort of manipulating the very limited human dopamine engine for porn?

Imagine just TikTok, but for porn. Yeah. That's like a brave new world.

I don't even know what it'll look like. Right? Like, again, you can imagine the behaviors of something smarter than you, but a super intelligent and and and agent that just dominates your intelligence so much will be able to completely manipulate you.

Is it possible that that it won't really manipulate. It'll just move past us. It'll just kinda exist the way water exists or the air exists.

You see? And that's the whole AI safety thing. It's not the machine that's gonna do that. It's other humans using the machine that are gonna do that to you.

Yeah. Because the machine is not interested in hurting humans. It's just

The machine is a machine. Yeah. But the human gets the machine, and there's a lot of humans out there very interested in manipulating you.

Well, let me bring up Alejes de Yatkowski. Who recently sat where you're sitting. He thinks that AI will almost surely kill everyone. Do you agree with him or not?

Yes, but maybe for a different reason.

Okay. And then I'll try to get you to find hope or we could find a note to that answer. But why?

Yes. Okay. Why didn't nuclear weapons kill everyone?

Good question.

I think there's an answer. I think it's actually very hard to deploy nuclear weapons tactically. Mhmm. It's very hard to accomplish tactical objectives great. I can nuke their country.

I have an irradiated pile of rubble. I don't want that. Why not? Why don't I want an irradiated pile of rubble? Yeah.

For all the reasons no 1 wants an irradiated pile of rumble.

Oh, because you can't use that land for for resources you can populate the land.

Yeah. What what you want AAA total victory in a war is not usually the air radiation and eradication of the people there. It's the subjugation and domination of the people. Mhmm.

Okay. So you can't use this strategically, tactically, in a war? Yeah. To help you to to help gain a military advantage. It's all complete destruction.

Alright? Yeah. But there's eagles involved, it's still surprising. It's still surprising that nobody pressed the big red button.

It's somewhat surprising, but You see, it's the little red button that's gonna be pressed with AI that's gonna you know, and that's why we die. It's it's not because the AI. If there's anything in the nature of AI, it's just the nature of humanity.

What's the algorithm behind the little red button? Well, like, what what what possible ideas do you have for the hollow human species ends?

Sure. So I think the most obvious way to me is wireheading. We end up amusing ourselves to death. We end up all staring at that infinite TikTok and forgetting to eat. Maybe maybe it's even more benign than this.

Maybe we all just stop reproducing. Now, to be fair, it's probably hard hard to get all of humanity.

Yeah. Yeah.

It

doesn't always go like, the the interesting thing about humanity is a diversity in

Oh, yeah.

You know, organisms in general. There's a lot of weirdos out there. Wow. 2 of them are sitting here.

Yeah. I I mean, diversity and humanity is We do respect. I wish I was more weird. No. Like, I'm kinda look, I'm drinking smart water, man.

That's like a Coca Cola product. Right? You won corporate. George Osborne. Corporate.

No. The amount of diversity in humanity, I think, is decreasing. Just like all the other biodiversity on the planet.

Oh, boy. Yeah. Right? And social media is not helping.

Go eat McDonald's in China. Yeah. No. It's the interconnectedness. That's that's that's that's doing it.

Oh, that's interesting. So everybody starts relying on the connectivity of the Internet and over time that reduces the diversity, the intellectual diversity. And then that gets you everybody into a funnel. There's still going to be a guy in Texas.

There is. And yeah. It's

a bunker.

To to be fair, do I think AI kills us all? I think AI kills everything we call like society today. But I do not think it actually kills the human species. I think that's actually incredibly hard to do.

You have a society, like, if we start over, that's tricky. Most of us don't know how to do most things.

Yeah. But some of us do. And they'll be okay and they'll rebuild after the Great AI. What's rebuilding look like? Like,

How much do we lose? Like, what is human civilization done? That's interesting. The combustion engine, electricity, So power and energy. That's interesting.

Like, how to harness energy. Whoa.

Whoa. Whoa. They're gonna be religiously against that.

Are they going to get back to, like, fire?

Sure. I mean, there'll be a there'll be a little bit, like, you know, some kind of amish looking kind of thing, I think. I think they're going to have very strong taboos against technology.

Like technology. It's almost like a new religion. Technology is the devil. Yeah. And nature is god.

Sure. So closer to nature. But can you really get away from AI? If it destroyed 99 percent of the human species, isn't it somehow have a hold, like a strong hold?

What's interesting about Everything we build, I think we are going to build superintelligence before we build any sort of robustness in the AI. We cannot build an AI that is capable of going out into nature and surviving like a like a bird. Right? A bird is an incredibly robust organism. We've built nothing like this.

We haven't built a machine that's capable of reproducing.

Yes. But there is, you know, a work of, like, robots a lot now. I have a bunch of them. They're mobile. Mhmm.

That can't reproduce, but all they need is, I guess, you're saying they can't repair themselves. If you have a large number, if you have like a hundred million of them,

let's just focus on them reproduce. Right? They have microchips in them. Mhmm. Okay.

Then do they include a fab? No. Then how are they gonna reproduce?

The other day, it doesn't have to be all on board. Right? They can go to a a factory, to a repair shop,

Yeah. But then you're really moving away from robustness. Yes. All of life is capable of reproducing without needing to go to a repair shop. Mhmm.

Life will continue to reproduce in the complete absence of civilization. Robots will not. So when if if the AI apocalypse happens, I mean, the AI's are gonna probably die out because I think we're gonna get again superintelligence long before we get robustness.

What about if you just improve the fab to where you you just have A3D printer that can always help you?

Well, that'd be very interesting. I'm interested in building that. Of course, you are.

You think how difficult is that problem? To have a robot that basically can build itself

very, very hard.

I think you've mentioned this like to me or somewhere where people think it's easy conceptually.

And then they remember that you're gonna have to have a fab.

Yeah. On board.

Of course.

So 3 d printer that print the 3 d printer.

Yeah.

Yeah. On legs.

Why

is that hard?

Well, because it's I mean, A3D printer is a very simple machine. Right? Okay. You're gonna print chips. You're gonna have an atomic printer.

How are you gonna dope the silicon?

Yeah.

Right? How are you gonna hatch the silicon?

You're gonna have to have a very interesting kind of fab if you wanna have a lot of computation on board. They can do, like, structural type of robots that are dumb.

Yeah. But structural type of robots aren't gonna have the intelligence required to survive in any complex environment.

What about like ants type of systems? We have like trillions of them.

I don't think this works. I mean, again, like, ants at their very core are made up of cells that are capable of individually reproducing.

They're doing quite a lot a lot of computation that we're taking for granted

It's not even just the computation. It's that reproduction is so inherent. Okay. So, like, there's 2 stacks of life in the world. There's the biological stack and the silicon stack.

The biological stack starts with reproduction. Reproduction is at the absolute core, the first proton RNA organisms were capable of reproducing. The silicon stack, despite as far as it's come, is nowhere near being able to reproduce.

Yeah. So the the the fab movement, digital fabrication. Fabrication in the full range of what that means is still in the early stages.

Yeah.

You're interested in this world.

Even if you did put a fab on the machine. Right? Let's say, okay, we can build fabs. We know how to do that as humanity. We can probably put all the precursors that build all the machines in the fabs also in the machine.

First off, this machine is gonna be absolutely massive. I mean, we almost have a, like, think of the size of the thing required to reproduce a machine today. Right? Like, is our civilization capable of reproduction? Can we reproduce our civilization on Mars?

If we were to construct a machine that is made up of humans, like a company, it can reproduce itself. Yeah. I don't know. I it feels like like like a hundred 15 people.

I get so much harder than that. Hundred and 20.

I just like

the phone number. I believe that Twitter can be run by 50 people. Uh-huh. I think that this is gonna take most of Like, it's just most of society. Right?

Like, we live in 1 globalized world.

No. But you're not interested in running Twitter. You're interested in seeding. Like, you want to seed a civilization and then because humans can, like,

Oh, okay. You're talking about yeah. Okay. So you're talking about the humans reproducing and, like, basically, like, what's the smallest self sustaining colony of humans? Yeah.

Yeah. Okay. Fine. But they're not gonna be making 5 nanometer chips.

Over time, they will I think you're being like, we have to expand our conception of time here. Come back to the original. Timescale. I mean, over across maybe a hundred generations, we're back to making chips. No.

If you seed the colony correctly,

maybe or maybe they'll watch our colony die out over here and be like, we're not making chips Don't make chips.

Nope. You have to seed that colony correctly.

Whatever you do, don't make chips. Chips are what led to their downfall.

Well, that is the thing that humans do. They they come up, the constructed devil, a good thing, and a bad thing, and they really stick by that, and then they murder each other over that. There's always gonna ask on the room for murders everybody. And he usually makes tattoos and nice branding.

That's what I see. Need that asshole. That's the question. Right. Humanity works really hard today to get rid of that asshole, but I think they might be important.

Yeah. This whole freedom of speech thing. It's it's the freedom of being in ASCO seems kind of important. Right. Man, this thing, this fab, this human fab that we've constructed, this human civilization is pretty interesting.

And now it's building artificial copies of itself or artificial copies of various aspects of itself that seem interesting like intelligence. And I wonder where that goes.

I like to think it's just like another stack for life. Like, we have like the BioStack life. Like, we're a BioStack life, another silicon stack life.

But it seems like the ceiling there might not be a ceiling and or at least the ceiling is much higher for the for the silcon stack.

Oh, no. I don't I we don't know what the ceiling is for the biostack either. The biostack The biostock just seem to move slower. You have Moore's Law, which is not dead despite many pro formations.

In the bio stack or the silicon?

In the silicon stack.

Mhmm. And

you don't have anything like this in the bio stack. So I have a meme that I I posted, I tried to make a meme. It didn't work too well. But I posted a picture of, you know, Ronald Reagan and and Joe Biden. And you look, this is 19 80 and this is 20 20.

Yeah.

And these 2 humans are basically like the same. Right? There's no there's no, like, there there's been no change in humans in the last 40 years.

Yeah.

And then I posted a computer from 19 80 and a computer from 20 20. Wow.

Yeah. With their early early stages. Right? Which is why you said when you said the fab, the size of the fab required to make another fab is like Very larger, I know.

Oh, yeah.

But computers were very large 80 years ago. And they got pretty tiny. Hence, there there people are starting to wanna wear them on their face. In order to escape reality, that's a thing in order to be live inside the computer. Yeah.

Put a screen right here. I don't have to see the rest of the US halls.

I've been ready for a long time.

You like virtual reality? I love it. You wanna live there? Yeah. Yeah.

Part of me does too. How far away are we, do you think?

Judging from what you can buy today far, very far.

I gotta tell you that I had the experience of Metas codec Avatar, where it's a ultra high resolution scam. It looked real.

I mean, the headsets just are not quite at like eye resolution yet. I haven't put on any headset or I'm like, oh, this could be the real world. Whereas when I put good headphones on, audio is there. Like, we we can reproduce audio that I'm like, I'm actually in a jungle right now. I I if I close my eyes, I can't tell them not.

Yeah. But then there's also smell and all that kind of stuff.

Sure.

I don't know. I the the power of imagination or the power of the the mechanism in the human mind that fills the gaps that kind of reaches and wants to make the thing you see in the virtual world real to you,

I believe in that power. Or humans wanna believe.

Yeah. Like, what what if you're lonely? What if you're sad? What if you're really struggling in life? And here's a world where you don't have to struggle anymore.

Humans wanna believe so much that people think the large language models are conscious. That's how much humans wanna believe.

Strong words. He's throwing left and right hooks. Okay. Why do you think large language models are not conscious?

I don't think I'm conscious.

Oh, so what is consciousness then? George Hartz?

It's like what it seems to mean to people, it's just like a word that atheists use for souls.

Sure. But that doesn't mean soul is not an interesting word.

If consciousness is a spectrum, I'm definitely way more conscious than the large language models are. I think the large language models are less contrast than the chicken.

When is the last time you've seen a chicken?

In Miami? Like, couple months ago?

How no. Like a living check.

Living check is walking around Miami. It's crazy.

Like, on the street? Yeah. Like a chicken.

A chicken. Yeah. Alright.

Alright. I was trying to call you all like like a good journalist and I I got shut down. Okay. But you don't think much about this kind of subjective feeling that it it feels like something to exist. And then as an observer, you can have a sense that an entity is not only intelligent, but as a kinda subjective experience of its reality, like a self awareness that is capable of, like, suffering, of hurting, of being excited by the environment in a way that's not merely kind of an artificial response, but a deeply felt 1.

Humans wanna believe so much if I took a rock and a sharpie and drew a sad face on the rock, they'd think the rock is sad.

Yeah. And you're saying, when we look in the mirror, we we apply the same smiley face with rock.

Pretty much. Yeah.

Doesn't it it's not weird, though, that you're not conscious.

Is that

No. But you do believe in consciousness. It's just an it's unclear. Okay. So to you, it's like a little, like, AAA symptom of the bigger thing That's not that important.

Yeah. It's interesting that, like, the human system seem to claim that they're conscious. And I guess they're trying to, like, says something in a straight up, like, okay. What do people when even if you don't believe in consciousness, what do people mean when they say consciousness? And there's definitely, like, meanings to it.

What's your favorite thing to

Pizza.

Cheese pizza. What are the toppings?

I like cheese pizza.

Don't say pineapple.

No. I don't like pineapple.

Okay. Pepperoni pizza.

Has they put any ham on it? Oh, that's real bad.

What's the best what's the best pizza? What are we talking about here? Like, do you like cheap crappy pizza? Chicago, deep dish, cheese, or Sure.

Oh, that's that's my favorite.

There you go. You bite into a deep dish, Chicago, deep dish pizza. And it feels like, so you were starving. You haven't eaten for 24 hours. You just bite in.

And you're hanging out with somebody that matters a lot to you. You're there. Pizza.

Sounds so nice.

Yeah. Alright. It feels like something. I'm I'm George, motherfucking hot eating. I'm fucking Chicago deep dish pizza.

There's just the full peak light living experience --

Yeah.

-- of being human, the top of the human condition.

Sure.

It feels like something to experience that.

Mhmm.

Why does it feel like something? That's consciousness. Isn't it?

If that's the word you wanna use to describe it, sure. I'm not gonna deny that that feeling exists. I'm not gonna deny that I experience that feeling. When I guess what I kind of take issue to is that there's some like like how does it feel to be a web server? Do 04:04 is hurt?

Not yet.

How would you know what suffering looked like? Sure. You can recognize a suffering dog because we're the same stack as the dog. All the biostacks stuff kind of, especially mammals, you know, it's it's really easy. You can

game recognize game.

Yeah. Versus the silicon stack stuff. It's like, you have no idea. You have You it don't Wow. The little thing has learned to mimic, you know.

But then I realized that that's all we are too. All of the little thing has learned to mimic.

Yeah. I guess, yeah, 444 could be could be suffering, but it's so far from our kind of living organism, our kind of stack. But it feels like AI can start maybe mimicking the biological stack. Buttery, buttery, buttery because it's strained.

Retrained it. Yeah.

And so in that maybe that's the definition of consciousness. Is the the biostat consciousness?

The definition of consciousness is how close something looks to human. Sure. I'll give you that 1.

No. How close something is to the human experience.

Sure. It's a very it's a very anthropocentric definition, but

where that's all we got.

Sure. No. And I I don't mean to like I think there's a lot of value in it. Look, I just started my second company. My third company will be AI girlfriends.

Oh, like I mean,

I wanna find out what your fourth company is.

Oh, wow.

Because I think once you have AI girlfriends, it's Oh, boy. Does it get interesting? Well, maybe let's go there. I mean, the relationships with AI that's creating human like organisms. Right?

And part of being human is being conscious, is being having the capacity to solve for having the capacity to experience this life richly in such a way that you can empathize. The AI is gonna empathize with you, and you can empathize with it, or you can project your anthropomorphic sense of what the other entity is experiencing, and and an AI model would need to you have to create that experience inside your mind. And it doesn't seem that difficult.

Yeah. But okay. So here's where it actually gets totally different. Right? When you interact with another human, you can make some assumptions.

Yeah. When you interact with these models, you can't. You can make some assumptions that that other human experiences, suffering, and pleasure in a pretty similar way to you do. The golden rule applies. Mhmm.

With an AI model, this isn't really true. Right? These these large language models are good at fooling people because they were trained on a a whole bunch of human data and told to mimic it.

Yep? But if if the AI system says, hi, my name is Samantha. It has a backstory.

Yeah.

I went to college here and there.

Here.

Maybe it'll it'll integrate this in AI system.

I made some chatbots. I gave them back stories. It was lots of fun. I was so happy when mama came out.

Yeah. We'll talk about love. We'll talk about all that, but, like, you know, the rock with this smiley face.

Yeah.

Why because it seems pretty natural for for you to anthropomorphize that thing. And then start dating it. And before you know it, you're married and have kids or the rock. Oh, the rock. This picture is on Instagram with you in the rock and smiley face.

To be fair, like, you know, something that people generally look for and they're looking for someone to date is intelligence and some form, and the rock doesn't really have intelligence. It's only a pretty desperate person with data rock.

I think we're all desperate deep down.

Oh, not rock level desperate. Alright.

Not rock level desperate, but AI level desperate. I don't know. I think all of us have a deep loneliness. It just feels like the language models are there. Oh, I agree.

And you know what, I won't even say this so cynically. I will actually say this in a way that like, I want a Ibrance. I do. Yeah. Like, I would love to You know, again, I the language models now are still a little like, people are impressed with these GPT things, and I look at, like or or like or the the co pilot, the coding 1.

And I'm like, okay. This is, like, junior engineer level, and these people are, like, fiverr level artists and copywriters. Like, okay, great. We got, like, fiber and, like, junior engineers. Okay, cool.

Like and this is just a start, and it will get better. Right? Like, I would I can't wait to have AI friends who are more intelligent than I am.

So fiber is just a temporary. It's not the ceiling.

No. Definitely not.

Is it is it count as cheating when you're talking to an AI model, emotional cheating?

That's that's up to you and your human partner to define.

Oh, you have to Alright.

You can Yeah. You you have to have that have to have that conversation. I guess.

Alright. I mean, integrate that with with porn and all this stuff.

No. I mean, this similar kind of porn.

Yeah.

Yeah. Right? I think people in relationships have different views on that.

Yeah. But most people don't have, like, serious open conversations about all the different aspects of what's cool and what's not. And it feels like AI is a really, weird conversation to have.

You know, the the the porn 1 is a good branching off. Sure. Like these things, you know, 1 of my scenarios that I put in my chatbot is I, you know, Nice girl named Lexi. She's 20. She just moved out to LA.

She wanted to be an actress, but she started doing only fans instead. And you're on a date with her. Enjoy.

Oh, man. Yeah. And so was that if you're actually dating somebody in in real life, is that cheating? I feel like it gets a little weird. Sure.

I think it's real weird. It's like, what are you allowed to say to an AI bot? Imagine having that conversation with a significant other

I mean, these are all things for people to define in their relationships. What it means to be human is just gonna start to get weird,

especially online. Like, how do you know? Like, there'll be moments when you'll have what you think is a real human you interact with on Twitter for years and you realize it's not.

I spread I love this meme. Heaven banning? Mhmm. You know what? Shadow banning.

Yeah. Alright. Shadow banning. Okay. You post and we can see Heaven banning.

You post. No 1 can see it, but a whole lot of AIs are spot up to interact with you.

Or maybe that's what the way human civilization ends is all of us a heaven band.

There's a great it's called my little pony friendship is optimal. Mhmm. It's a sci fi story that explores this idea. Friendship is optimal.

Yeah. I'd like to have some at least on the intellectual realm, some AI friends. Yeah. That argued with me. But the the the romantic realm is weird.

Definitely weird. But not out of the realm of the the kind of weirdness that human civilization is capable of, I think. I think

I want it. Look, I want it. If no 1 else wants it, I want it.

Yeah. I think a lot of people probably

want it.

There's a deep loneliness.

And I'll fill their loneliness and, you know, it just will only advertise to you some of the time.

Yeah. Maybe the conceptions of noggin me changed to. Like, I grew up in a time, like, I value monogamy, but maybe that's a silly notion when you have arbitrary number of AI systems.

This this this interesting path from rationality to polyamory. Yeah. That doesn't make sense for me.

For you. But you're just a biological organism who was born before, like, re the Internet really took off.

The crazy thing is, like, culture is whatever we define it as. Right? These things are not easy. Like, is a problem and moral philosophy Right? There's no, like like, okay.

What is might be that, like, computers are capable of mimicking, you know, girlfriends perfectly. They passed the girlfriend during test. Right? But that doesn't say anything about ought. That doesn't say anything about how we ought to respond to them as a civilization.

That doesn't say we ought to get rid of monogamy. Right? That's a completely separate question, really a religious 1.

A girlfriend touring test. I wonder what that looks like.

A girlfriend touring test.

Are you writing that Will you be the the the Allan touring of the 20 first century? That writes the the girlfriend touring test.

No. I mean, of course, my my hey, I have girlfriends. Their goal is to pass the girlfriend current test.

Nobody there should be like a paper that kind of defines the test. Or you so, I mean, the question is if it's deeply personalized over there is a common thing that really gets everybody. Yeah.

I mean, you know, look, we're a company. We don't have to get everybody. We just have to get a large enough client outstanding.

Like how you're already already thinking company. Alright. Let's before we go to company number 3 and company number 4, let's go to company number 2. Alright. Tiny Corp.

Possibly 1 of the greatest names of all time for a company. You've launched a new company called Tiny Corp. That leads the development of Tiny GRAD. What's the origin story of Tiny Corp. And Tiny GRAD?

I started tiny grad as a, like, a toy project just to teach myself. Okay. Like, what is a convolution? What are all these options you can pass to them? What is the derivative of evolution?

Right? Very similar to Carpathia road micro grad. Very similar. And then I started realizing I started thinking about, like, AI chips. I started thinking about chips that run AI and I was like, well, okay, this is going to be a a really big problem.

If Nvidia becomes a monopoly here, How long before NVIDIA is nationalized?

Mhmm. So you 1 of the reasons that Star tiny Corp is to challenge Nvidia.

It's not so much To challenge Nvidia. I I actually I I like Nvidia and it's to make sure power stays decentralized.

Yeah. And here's computational power.

Mhmm.

And to you, Nvidia is kinda locking down the computational power of the world.

If Nvidia becomes just like 10 x better than everything else, you're giving a big advantage to somebody who can secure NVIDIA as a resource. Yeah. In fact, If Jenson watches his podcast, he may wanna consider this. He may wanna consider making sure his company is not nationalized.

Do you think that's an actual threat?

Oh, yes.

No. But there's so much you know, there's AMD.

Mhmm. So we have Nvidia and AMD. Great.

Alright. But you don't you don't think there's, like, a push towards, like, selling, like, Google selling GPUs or something like this, you don't think there's a push for that?

Have you seen it? Google loves to rent you GPUs.

It doesn't you can't buy it or bus buy? No.

So I started to work on a on chip. I was like, okay. What is it gonna take to make a chip? And my first notions were all completely wrong, about why, about like how you can improve on GPUs. And I will take this.

This is from Jim Keller on your podcast. Mhmm. And this is 1 of my absolute favorite descriptions of computation. So there's 3 kinds of computation paradigms that are common in the world today. Other CPUs, and CPUs can do everything.

CPUs can do add and multiply, They can do load in store, and they can do compare and branch. Mhmm. And when I say they can do these things, they can do them all fast. Right? So compare and branch are unique to CPUs, what I mean by they can do them fast is they can do things like branch prediction and speculative execution, and they spend tons of transistors on these super deep reorder buffers in order to make these things fast.

Then you have a simpler computation model GPUs. GPUs can't really do compare and branch. I mean, they can, but it's horrendously slow. But GPUs can do arbitrary load and store. Right?

GPUs can do things like x, d reference y. So they can fetch from arbitrary pieces of memory. They can fetch from memory that is defined by the contents of the data. The third model of computation is DSPs. And DSPs are just add and multiply.

Right? Like, they can do load in stores, but only static load in stores. Only loads of stores that are known before the program runs. And you look at neural networks today and 95 percent of neural networks are all the DSP paradigm. They are just statically scheduled ads and multiplies.

So tiny guys really took this idea and and I'm still working on it to extend this as far as possible. Every stage of the stack has turned completeness. Right? Python has turned completeness. And then we take python, we go to c plus plus, which is turned late, and then maybe c plus plus calls into some CUDA kernels, which are turned complete.

The CUDA kernels go through LVM, which is and complete into PTX, which is turn complete, SaaS, which is turn complete, on a turn complete processor. I wanna get turn completeness out of the stack entirely. Because once you get rid of turn completeness, you can reason about things. Rises theorem in the halting problem do not apply to add more machines.

Okay. What's the power and the value of getting to and complete this out of? Out of are we talking about the hardware or or the software?

Every layer of the stack. Every layer of the stack, removing turn completeness allows you to reason about things. Right? So the reason you need to do branch prediction in a CPU, and the reason it's prediction and the branch predictors are I think they're, like, 99 percent on CPUs. Why do they get 1 percent of them wrong?

Well, they get 1 percent wrong because you can't know. Right? That's the halting problem. It's equivalent to the halting problem to say whether a branch is gonna be taken or not. Mhmm.

I can show that. But the admiral machine, the neural network runs the identical compute every time. The only thing that changes is the data. So when you realize this, you think about, okay, how can we build a computer How can we build a stack that takes maximal advantage of this idea? Mhmm.

So what makes tiny grid different from other neural network libraries? Is it does not have a preventative operator even for matrix multiplication. Right? And this is every single 1. They even have preventative operators.

There's things like convolutions.

So no Matt Moll.

No Matt Moll. Well, here's what Matt Moll is, so I'll use my hands to talk here. Mhmm. So if you think about Cube, and I put my 2 matrices that I'm multiplying on 2 phases of the cube. Right?

You can think about the matrix multiply as, okay, The n cubed, I'm gonna multiply for each 1 in the cubed, and then I'm gonna do a sum which is a reduce up to here to the third phase of the cube and that's your multiplied matrix. So what a matrix multiply is, is a bunch of shape operations. Right? A bunch of permute 3 shapes and expands on the 2 matrices. A multiply and cubed.

A reduce and cubed, which gives you an n squared matrix.

Okay. So what what is the minimum number of operations that can accomplish that if you don't have that mall is a primitive.

So tiny grad has about 20. And you can compare tiny grad's offset or IR to things like XLA. Or prim torch. So excellent and prim torch are ideas where, like, okay, torch has, like, 2000 different kernels. Mhmm.

Pi Torch 2 introduced prim Torch, which has only 250. Tiny grad has order of magnitude 25. It's it's 10 x less than XLI or prem George. And you can think about it as kind of like risk versus risk. Right?

These other things are risk like systems. Tanya graders risk. Risk 1. Risk architecture is gonna change everything. 19 95, hackers.

Wait, really? That's an actual thing.

Angelina Jolie delivers the line. Risk architecture is gonna change everything in 19 95. And here we are with arm and the phones. And arm everywhere.

Wow. I love it when movies actually have real things in them. Right. Okay. Interesting.

So this is like so you're thinking of this as the risk architecture of ML stack. 25. What what can you can you go through the the 4 op types?

Sure. Okay. So you have unary ops which take in a tensor and return a tensor of the same size. And do some urinary optuit. X, log, reciprocal, sign.

Right? They've taken 1 in their point wise.

Mhmm. Really?

Yeah, really. Almost all activation functions are unary ops. Some combinations of unary ops together is still a unary op. Mhmm. Then you have binary ops.

Binary ops are like pointwise addition, multiplication division, compare. It takes in 2 tensors of equal size. And outputs 1 tensor. Mhmm. Then you have reduce ops.

Reduced ops will, like, take a three-dimensional tensor and turn it into a 2 dimensional tensor. Or three-dimensional temperature turner to 0 dimensional temperature. Think like a sum or max are really the common ones there. And then the 4 type is movement tops. And moving to apps are different from the other types because they don't actually require computation.

They require different ways to look at memory. Mhmm. So that includes reshapes, permutes, expands, flips. Those are the main ones. Probably

And so with that, you have enough to make a map model.

And convolutions. And every convolution you can imagine dilated convolutions, strided convolutions, transposed convolutions.

You're right on GitHub about laziness. Showing a map map. Matrix multiplication. See how despite the style is fused into 1 kernel with the power of laziness. Can you elaborate on this power of laziness?

Sure. So if you type in Pytorch a times b plus c, What this is going to do is it's going to first multiply add and b, and store that result into memory. And then it is going to add c by reading that result from memory, reading c from memory and writing that out to memory. There is way more loads the stores to memory than you need there. If you don't actually do a times b, as soon as you see it, if you wait until the user actually realizes that tensor, until the laziness actually resolves, you confuse that plus c.

This is, like, it's the same way has to all works.

So what's the process of porting a model into tiny grid?

So tiny red's front end looks very similar to Pytorch. I probably could make a perfect or pretty close to perfect interop layer if I really wanted to. Think that there's some things that are nicer about tiny grad syntax than Pytorch, but the front end looks very torch like. You can also load in Onyx models. Okay.

We have more on x test passing than core AML? Core

AML. Oh, okay. So We'll

we'll pass on x span time soon.

What what about, like, the developer experience with tiny grad? I mean, what it feels like, what a versus Pytorch?

By the way, I really like Pytorch. III think that it's actually a very good piece of software. I think that they've made a few different trade offs. And these different trade offs are where, you know, Time to grab, text it from a path. 1 of the biggest differences is it's really easy to see the kernels that are actually being sent to the GPU.

Right? If you run PyTorch on the GPU, you, like, do some operation and you don't know what kernels ran. You don't know how many kernels ran. You don't know how many flops were used. You don't know how much memory access is reused, tiny grad type debug equals 2.

And it will show you in this beautiful style, every kernel that's run. How many flops? And how many bytes?

So can you just linger on what problem tiny grad solves.

Tiny grad solves the problem of porting new ML accelerators quickly. 1 of the reasons tons of these companies now. I think Sequoia marked Graph Core to 0. Right? Saravis, tensorrent, grok.

All of these ML accelerator companies. They built chips. The chips were good. The software was terrible.

Mhmm.

And part of the reason is because but I think the same problem is happening with dojo. It's really, really hard to ride a pie torch port. Mhmm. Because you have to write 250 kernels and you have to tune them all for performance.

What does Jim? Jim color think about Tanya Grant? You guys hung up quite a bit. So he's, you know, he's he was involved. He's involved with brainstorm.

What's his praise and what's this criticism of what you're doing with your life.

Look, my prediction for tense torrent is that they're gonna pivot to making risk 5 gyps. CPUs.

CPUs. Yeah. Why? Why? Because

AI accelerators are a software problem, not really a hardware problem.

Oh, which things that you don't think you you think the diversity of AI accelerators in the hardware base is not going to be a thing that exists long term.

I think what's gonna happen is if I can fin it. Okay. If you're trying to make an AI accelerator, you better have the capability of writing a torch level performance stack on NVIDIA GPUs. If you can't write a torch stack on NVIDIA GPUs, and I mean all the way, I mean down to the driver, there's no way you're gonna be able to write it on your chip because your chip's worse than an NVIDIA GPU. The first version of the chip you tape out is definitely worse.

Oh, you're saying wearing that? That's really tough.

Yes. And not only that, actually, the chip that you tape out, almost always because you're trying to get advantage over NVIDIA, you're specializing the hardware more. It's always harder to write software for more specialized hardware. Like a GPU is pretty generic. And if you can't write an NVIDIA stack, there's no way you can write a stack for your chip.

So My approach to tiny grad is first, write a performant NVIDIA stack. We're we're targeting AMD.

So you did say a few to NVIDIA a little bit. Would love.

With love.

Yeah. With love.

So what are the yankees? You know? Oh, Matt's fan.

Oh, you're you're you're a Matt's fan. A risk a risk fan and a mass fan. What's the hope that AMD has? And and you did build with AMD recently that I saw. How does the the the 7900 XTX compared to the RTX 40 90 or 40 80?

Well, let's start with the fact that the 7900 XTX kernel drivers don't work. And if you run demo apps in loops, it panics the kernel.

Okay? So if this is a software issue,

Lisa who responded to my email. Oh, I reached out. I was like, this is, you know, really? Like, I understand if you're 7 by 7 transposed WinAgrad Calm is slower than NVIDIA's. But literally, when I run demo apps in a loop, the kernel panics.

So just adding that loop.

Yeah. I just I just literally took their demo apps and wrote, like, wild true semi colon do the app semi calling done in a bunch of screens.

Mhmm.

Right? This is, like, the most primitive fuzz testing.

Why do you think that is they're just not seeing a market in the in machine learning,

they're changing. They're trying to change. They're trying to change. And I had a pretty positive interaction with them this week. Last week, I I went on YouTube, I was just like, that's it.

I give up on AMD. Like, this is the their driver doesn't even like, I'm not gonna I'm not gonna, you know, I'm I'll I'll go with Intel GPUs. Right? Until GPUs have better drivers.

So you're kinda spearheading the diversification of GPUs.

Yeah. And I'd like to extend that diversification to everything. I'd like to diversify the, right, the more My central thesis about the world is there's things that centralize power and they're bad and there's things that decentralize power and they're good. Everything I can do to help decentralize power I'd like to do.

So you're really worried about the centralization of NVIDIA. Isn't interesting. And you don't have a fundamental hope for the the proliferation of ASICs, except in the cloud.

I'd like to help them with software. No. Actually, there's only the only ASIC that is remotely successful is Google's CPU.

Mhmm.

And the only reason that's successful is because Google wrote a machine learning framework. Alright. I I think that you have to write a competitive machine learning framework in order to be able to build an ASIC.

You think meta with Pytorch builds competitor? I hope so.

Okay. They have 1. They have an internal 1.

Internal. I mean, public facing with a nice cloud and interface and so on.

I don't want a cloud.

You don't like cloud?

I don't like cloud.

What do you think is the fundamental limitation of cloud?

Fundamental limitation to cloud is who owns the the the off switch?

So it's the power to the people. Yeah. And you don't you don't like the man to have all the power. Exactly.

Alright.

And right now, the only way to do that is with the NBA GPUs if you want performance and stability. Interesting. It's a it's a costly investment emotionally to go with AMD's Well, let me add sort of an attention to ask you. What what you've built quite a few p c's. What's your advice on how to build a good custom p c?

For, let's say, for the different applications that you use, for gaming, for machine learning.

Well, you shouldn't build 1. You should buy a box from the tiny corp.

I heard rumors, whispers about this box in the tiny corp. What's what's this thing look like? What is it what is it called?

It's called the tiny box. Tiny box. It's 15000 dollars. Yeah. And it's almost a paid a flop of compute.

It's over a hundred gigabytes of GPU RAM. It's over 5 terabytes per second of GPU memory bandwidth. Gonna put, like, 4 NVMe's in in in raid. You're gonna get, like, 20, 30 gigabytes per second of drive read bandwidth. I'm gonna I'm gonna build like the best deep learning box that I can that plugs into 1 wall outlet.

Okay. Can you go to the specs again a little bit from your from memory?

Yeah. So it's almost a paid off of compute.

So in D and L,

Today, I'm leaning toward AMD. Mhmm. But we're pretty agnostic to the type of compute. The the the main limiting spec is a 120 volt 15 amp circuit.

Okay.

Well, I mean it. Because in order to, like like, there's a plug over there.

Mhmm. Alright?

You have to be able to plug it in. We're also gonna sell the tiny rack, which like what the most power you can get into your house without a rousing suspicion. And 1 of the 1 of the answers is an electric car charger.

Where where does the rack go?

Your garage?

Interesting. The car charger.

A wall outlet is about 1500 watts. A car charger is about 10000 watts.

What is the most amount of power you can get your hands on without a rousing suspicion?

That's right.

George Hoss. Okay. So the the tiny box and you said NVMe is in raid. I forget what you said about memory, all that kind of stuff. Okay.

So what about what GPUs?

Again, probably Probably 7900 x t x's, but maybe 30 nineties, maybe a 7 seventies.

Because sometimes you're flexible or still exploring.

I'm still exploring. I wanna I wanna deliver a really good experience to people

-- Mhmm. --

and yeah, what GPUs I end up going with again? I'm leaning toward AMD. We'll see. You know, in my email, what I what I did to AMD is like, Just dumping the code on GitHub is not open source. Open source is a culture.

Open source means that your issues are not all 1 year old stale issues. Open Source means developing in public. And if you guys can commit to that, I see a real future for AMD as a competitor to Nvidia.

Well, I'd love to get a tiny box to MIT. So whenever it's ready.

What was that?

Let's do it.

We're taking preorders. I took I took this from Milan. I'm like, alright. Hundred dollar fully refundable preorders.

Is it gonna be, like, the cyber truck that's gonna take a few years? Or

No. I'll try and do it faster. It's a lot simpler than the truck.

Well, there's complexities not just the putting the thing together, but, like, shipping and all this kind of stuff.

The thing that I wanna deliver to people out of the box is being able to run 65000000000 parameter llama in f p 16. In real time. And, like, are good. Like, 10 tokens per second or 5 tokens per second or something?

Just it works. Yep. John was running or something like Llama. Experia

or I think Falcon is is is the new 1. Experience a chat with the largest language model that you can have in your house.

Yeah. From from a wall plug.

From a wall plug. Yeah. Actually, for inference, it's not like even more power would help you. Get more.

Even my boss wouldn't get you more.

Well, there's just the biggest the biggest model released is is 65000000000 primary along as far as I know.

So it sounds like tiny box will naturally pivot towards company number 3 because you could just get the girlfriend and or boyfriend.

That one's harder, actually.

The boyfriend is harder?

The boyfriend's harder. Yeah.

I think that's a very biased statement. I know a lot of people just what's what why is it harder to replace a boyfriend than a other girlfriend with the artificial LLM? Because women are attracted

to status and power and men are attracted to youth and beauty. No. I mean, this is what I mean. But what Both are it

could be amimicable, easy to the language model.

No. No machines do not have any status or real power.

I don't know. I think you both well, first of all, you're using language mostly to to communicate youth and beauty and power and status.

But status fundamentally is here some game. Alright. Whereas youth and beauty or not?

No. I think status is a narrative you can construct. I I don't think status is real.

I don't know. I just think that that's why it's harder. You know, yeah, maybe it is my biases.

I think status is way easier to fake.

I also think that, you know, men are probably more desperate and more likely to buy my products, so maybe they're a better target market.

Despiration is interesting. Easier to fool. That's I could I could see that.

Yeah. Look. I mean, look. I know you can look at porn viewership numbers. Right?

A lot more men watch porn than women. Yeah. That's why that is.

There's a lot of questions and answers you can get there. Anyway, with the with the tiny box, How many GPUs in tiny box? Sixth. Oh, man.

I'm and I'll tell you why it's 6.

Yeah.

Uh-huh. So AMD epic processors have a hundred 28 lanes of PCIe. Mhmm. I wanna leave enough lanes for some drives. Mhmm.

And I wanna leave enough lanes for some networking.

How do you do cooling for something like this?

That's 1 of the big challenges. Not only do I want the cooling to be good, I want it to be quiet. I want the tiny box to be able to sit comfortably in your room.

Right? This is really going towards the girlfriend thing. Because because you want to run the LMM?

I'll give I'll give a more I mean, I can talk about how it relates to company number 1. Come AI. Yeah. Well,

but, yes, quiet. Oh, quiet because you may be potentially wanna run-in a car?

No. No. Quiet because you wanna put this thing in your house. And you want it to coexist with you. If it's streaming, it's 60 d b, you don't want that in your house.

You'll kick it out.

60 d b. Yeah.

I want, like, 40, 45.

So how do you make the cooling quiet that's an interesting problem in itself.

A key trick is to actually make it big ironically. It's called the tiny box. Yeah. But if I can make it big, a lot of that noise is generated because of high pressure air.

Mhmm. If

you look at like A1U server, A1U server has these super high pressure fans. They're like super deep and they they're like Greenwich. Versus if you have something that's big. Well, I can use a big and, you know, you know, they call them big ass fans. Those ones that are, like, huge in the ceiling.

And they're completely silent.

So tiny box

will be big. It is the I do not want it to be large according to UPS. Want it to be shippable as a normal package, but that's my constraint there.

Interesting. Well, the the fan stuff, like, can't can't it be assembled on location or

No. That should be. Well, here you're here. Look, I wanna give you a grade out of the box experience. I want you to lift this thing out.

I want it to be like like the Mac, you know. Tiny box.

The Apple experience. Yeah. I love it. Okay. And so tiny box would run tiny grad.

Like, what what what do you envision this whole thing to look like? We're talking about, like, Linux with a full software engineering environment.

Mhmm.

And it's just not PyTorch, but tiny grid.

Yeah. We did a poll if people want you bun to our arch. We're gonna stick with you bun to.

Oh, interesting. What's your favorite flavor of this? Kupantamate. How do we pronounce that meat? So how do you you've gotten llama into tiny grad.

You've gotten stable diffusion into tiny grad. What was that like? Can you comment on, like, what are what are these models? What's interesting about porting them? Sure.

It's yeah. Like, what what are the the challenges? What are what's naturally, what's easy, all that kind of stuff.

There's a really simple way to get these models into tiny grad, and you can just export them as Onix. Mhmm. And then Honey grad can run Onix. So the ports that I did of Llama, Stable Defusion, and now Whisper, are more academic to teach me about the models. But they are cleaner than the Pirate versions.

You can read the code. I think the code is easier to read. It's less lines. There's just a few things about the way tiny God writes next. Here's Here's a complaint I have about Pytorch.

N n dot relu is a class. Right? So when you create a when you create an NN module, you'll put your NN relu's as in a nip. And this makes no sense. Relay was completely stateless.

Why should that be a class?

But but that's more like a software engineering thing. Or do you think it has a cost on performance?

Oh, no. It doesn't have a cost on performance. But, yeah, no. I I think that it's that's what I mean about, like, tiny grads front end to being cleaner.

I see. What do you think about Mojo? I don't know if you've been paying attention in the programming language that does some interesting ideas that kinda intersect tiny grid.

I think that there's a spectrum. And, like, on 1 side, you have mojo. And on the other side, like, GGML. Mhmm. GGML is this, like, we're gonna run llama fast on Mac.

Mhmm. Okay. We're gonna expand out to a little bit, but we're gonna basically go to depth first. Right? Mojo was like, we're gonna go breathe first.

We're gonna go so wide that we're gonna make all of Python fast

--

Mhmm. -- and tiny words in the middle. Again, we are going to make neural networks fast.

Yeah. But they they try to really get it to be fast, compiled down to specifics hardware, and make that compilation step. As flexible and resilient as possible,

and but they have turn completeness.

And that will miss you. Turn, that's what you're saying. It's somewhere in the middle. So you're actually going to be targeting some accelerators, some, like, some some number, not 1.

My goal is step 1, build an equally performance stack to PyTorch on Nvidia and AMD. Mhmm. But with way less lines. And then step 2 is, okay, how do we make an accelerator. Right?

But you need step 1. You have to first build the framework before you can build the accelerator.

Can you explain m l perf What's your approach in general to benchmark in tiny grad performance? So

I'm much more of a like build it the right way and worry about performance later. There's a bunch of things where I haven't even, like, really dove into performance. The only place where tiny grad is competitive performance wise right now is on Qualcomm GPUs.

Mhmm.

So tiny grads actually use an open palette, try the model. So the driving model is is is tiny grad.

And when did that happen? That transition?

About 8 months ago now? And it's 2 x faster than Qualcomm's library.

What's the hardware open that open pilot runs on the the the Kamai app?

It's a Snapdragon 08:45.

Okay. So this

is using the GPU. So the GPU is an adreno GPU. There's like different things. There's like really good Microsoft paper that talks about, like, mobile GPUs and why they're different from desktop GPUs.

Mhmm.

1 of the big things is in a desktop GPU, you can use buffers. On a mobile GPU image textures a lot faster.

On a mobile GPU image textures and limit okay. And so you want to be able to leverage that

I wanna be able to leverage it in a way that it's completely generic. Right? So there's a lot of this Xiaomi has a pretty good open source library for mobile GPUs called Mace, where they can generate where they have these kernels, but they're all hand coded. Right? So that's great if you're doing 3 by 3 comps.

That's great if you're doing dense map models. But the minute you go off the beaten path at tiny bit, well, your performance is nothing.

Since you mentioned an open pilot, I'd love to get an update in the company number 1, com AI world, how are things going there and the development of semi autonomous driving?

You know, almost no 1 talks about FSD anymore and even less people talk about open pilot? We've solved the problem. Like, we solved it years ago.

What's the problem exactly? Well, how do you Like, what what what does solving it mean?

Solving means how do you build a model that outputs a human policy for driving? Mhmm. How do you build a model that given you know, reasonable set of sensors outputs a human policy for driving. So you have, you know, companies like Wehman Cruises, which are hand coding, these things that are, like, quasi human policies. Then you have Tesla and maybe even to more of an extent, comma, asking, okay, how do we just learn human policy and data?

The big thing that we're doing now and we just put it out on Twitter, At the beginning of comma, we published a paper called learning a driving simulator. Mhmm. And the way this thing worked, was it's a it was an auto encoder, and then an overnight in the middle. Right? You take an auto encoder?

You compress the picture, use an r and n, predict the next state, and these things were, you know, it was a laugh at the bad simulator. This is 20 15 hour machine learning technology. Today, we have VQVAE, and transformers. Mhmm. We're building DriveGPT, basically.

DriveGPT. Okay. So and then it's trained on what? Is it trained in a self supervised way?

Yeah. It's trained on all the driving data to predict the next frame.

So really trying to learn a human policy. What would a human do?

Well, actually, our simulator's condition on the pose. It's it's actually a simulator. You can put in like a state action pair and get out the next state. Okay. And then once you have a simulator, you can do RL in the simulator, and RL will get us that human policy.

So transfers.

Yeah. RL with a reward function, not asking, is this close to the human policy, but asking, would a human disengage if you did this behavior?

Okay. Let me think about the the distinction there. Would a human disengage? Yeah. Would a human disengage that correlates, I guess, with the human policy, but it can be different.

So it's it's it doesn't just say, oh, what a human do, it says, what would a good human driver do? Yeah. And such that the experience is comfortable, but also not annoying in that, like, the thing is very cautious. It's in that if finding a nice balance. That's that's interesting.

It's in nice.

It's asking exactly the right question. What will make our customers happy? Right. A system that you never wanna disengage.

Because usually, this engagement is almost always a sign of I'm not happy with what the system is doing.

Usually, there's some that are just I felt like driving, and those are always fine too, but they're just gonna look like noise in the data.

But even that felt like driving. Maybe yeah. That's even that's a signal. Like, why do you feel like driving? You're you need to recalibrate your relationship with a car.

Okay. So what that that's really interesting. How close are we just solving self driving?

It's hard to say. We haven't completely closed the loop yet. So we don't have anything built. That truly looks like that architecture yet. Mhmm.

We have prototypes and bugs. So we are A couple bug fixes away. Might take a year. Might take 10.

What's the nature of the bugs? Are these these major philosophical bugs, logical bugs? What kind of what kind of bugs are we talking about?

They're just like they're just like stupid bugs and, like, also, we might just need more scale. We just massively expanded our compute cluster, a comma. We now have about 2 people worth of compute, 40 pin flaps.

Well, people people are different.

Yeah. I have 20 faith floss. That's a person. I mean, it's just it's just a unit. Right?

Horse's are different too, but we still call it a horsepower.

Yeah. But there's something different about mobility than there is about perception and action in a very complicated world. But yes.

Well, yeah. Of course, not all flops are created equal. If you have randomly initialized weights, it's not gonna

Not all flops are created equal.

So we're just doing way more useful things than others.

Yep. Yep. Tell me about it. Okay. So more data.

Scale means more scale in computer or scale in scale of data.

Both

diversity of data.

Diversity is very important in data. Yeah. I mean, we have so we have about I think we have, like, 5000 daily actives.

How would you evaluate how FSD is doing? There's life driving.

Pretty well.

How's that race going? Between Commeister and F50?

Tesla is always 1 to 2 years ahead of us. They've always been 1 to 2 years ahead of us. And they probably always will be because they're not doing anything wrong.

What have you seen that since the last time we talked that are interesting architectural decisions, training decisions, like the way the way they deploy stuff, the architectures they're using in terms of the software, how the teams are running, all that kind of stuff, data collection. Anything interesting?

I mean, I know they're moving toward more of an end to end approach. So

creeping towards end to end as much as possible across the whole thing. The training, the data collection, and everything.

They also have a very fancy simulator. They're probably saying all the same things we are. They're probably saying we just need to optimize, you know, what is the reward? Well, you get negative reward for disengagement. Right?

Like, everyone kinda knows this. It's just a question who can actually build and deploy the system.

Yeah. I mean, this good this requires good software engineering, I think. Yeah. And and the right high enough hardware.

Yeah. The hardware to run it.

You still don't believe in cloud in that regard?

I have a compute cluster in my boss. 800 amps?

Tiny grad.

With 40 kilowatts at idle, our data center. Dyson crazy. With 40 kilowatts just burning just when the computers are idle.

Just 1 moment.

Sorry. Sorry. Compute cluster.

Compute cluster. I got it.

It's not a data center. Yeah. Now data centers are clouds. We don't have clouds. Data centers have air conditioners.

We have fans. That makes it a compute cluster.

I'm guessing this is a kind of a legal distinction of sure. Yeah.

We have a compute cluster.

You said that you don't think LOMs have consciousness or at least not more than chicken? Do you think they can reason? Is there something interesting to you about the word reason? About some of the capabilities that we think is kind of human to be able to integrate complicated information and through a chain of thought arrive at a conclusion that feels novel. A novel integration of the of disparate facts

Yeah, I I don't think that there's I think if I can reason better than a lot of people.

Hey, isn't that amazing to you though? Is that like an incredible thing that a transform could achieve?

I mean, I think that calculators can add better than a lot of people.

But language feels like reasoning through the process of language, which looks a lot like thought.

Making brilliant season chess, which feels a lot like thought. Like, whatever new thing that AI can do, everybody thinks is brilliant. And then, like, 20 years go by and they're like, well, you have chest. That's like mechanical. Like adding, that's like mechanical.

So using language is not that special. It's like chess.

It's like chess. I

don't know. And because it's very human, we we take it we listen, there is something different between chess and language. Chest is a game that a subset of population plays. Language is something we use nonstop for all of our human interaction. And human interaction is fundamental to society.

So it's like, holy shit. This this language thing is not so difficult to, like, create a new machine.

The problem is if you go back to 19 60 and you tell them that you have a machine that can play amazing chess. Of course, someone in 19 60 will tell you that machine is intelligent, someone in 20 10 won't. What's changed? Right? Today, we think that these machines that have language are intelligent.

But I think in 20 years, we're gonna be like, yeah, but can it reproduce?

So reproduction. Yeah. We might redefine what it means to to be what is it? A high performance living organism on earth.

Humans are always gonna define a niche for themselves. Like, well, you know, we're better than the machines because we can you know, and like they tried creative for a bit, but no 1 believes that 1 anymore.

But Nish, is is that is that delusional or is there some accuracy to that? Because maybe, like, with chess, you start to realize, like, that that we have ill conceived notions of what what makes human special, like the apex organism on earth.

Yeah. And I think maybe we're gonna go through that same thing with language. And that same thing with creativity.

The language carries these notions of truth and so on. And so we might be like, wait, maybe truth is not carried by language. Maybe there's like a deeper thing.

The niches getting smaller.

Oh, boy.

But no. No. No. You don't understand humans are created by God and machines are created by humans, therefore. Right?

Like, that'll be the last issue we have.

So what do you think about this the rapid development of LMs? If you could just, like, stick on that. It's still incredibly impressive, like, with JD BPT. Just even JD BPT, what are your thoughts about reinforcement learning with human feedback on these large language models?

I'd like to go back to when calculators first came out and or computers. And like, I wasn't around, look, I'm I'm 33 years old. And to like see how that affected. Like, society.

Maybe you're right. So I wanna put on the the the big picture hat here.

I got a refrigerator. Wow.

Refrigerator, electricity, all that kind of stuff. But, you know, with the Internet, large language models seeming human like basically passing a Turing

test. Mhmm.

It seems it might have really at scale rapid transformative effects on society. But you're saying like other technologies have as well. So maybe calculator is not the best example of that. Because that just seems like well, no, maybe, calculator.

For milk man, the day he learned about refrigerators, he's like, I'm done. You're telling me you just keep the milk in your house. You don't even need to deliver it every day I'm done.

Well, yeah, that you have to actually look at the practically impacts of certain technologies. That that they've had, yeah, probably electricity is a big 1 and also how rapidly it's spread. Yeah. The Internet is a big 1.

How do you think it's different this time, though?

It just feels like

The niches getting smaller.

The niches humans --

Yes.

-- that makes humans special. Yes. It feels like it's getting smaller rapidly though. Doesn't it? Or is that just a feeling we dramatize everything?

I think we dramatize everything. I think that that that asked the milk man when he saw refrigerators. And they're gonna have 1 of these in every home.

Yeah. Yeah. Yeah. Yeah. May but boys are impressive.

So much more impressive than seeing a a chess world champion AI system.

I disagree, actually. I disagree. I think things like u 0 and alpha go are so much more impressive because these things are playing beyond the highest human level. The language models are writing middle school level essays and people are like, wow, It's a great essay. It's a great 5 paragraph essay about the causes of the civil war.

Okay. Forget the civil war, just generating code, codex. So you're you're saying it's mediocre code. Terrible. But I don't think it's terrible.

I think it's just mediocre code. Yeah. Often close to correct, like, for medial curve.

That's the scariest kind of code. I spent 5 percent of time typing and 95 percent of time debugging. The last thing I want is close to correct code. I want a machine that can help me with the debugging now with typing.

You know, it's like l 2 level to driving, similar kind of thing. Yet, it's you still should be a good programmer in order to modify. I wouldn't even say debugging. Just modifying the code, deleting it.

Don't think it's like level 2 driving. I think driving is not tool complete and programming is. Meaning, you don't use, like, the best possible tools to drive. Right. You're not you're not like like like cars have basically the same interface for the last 50 years.

Yep. Computers have a radically different interface.

Okay. Can you describe the concept of tool complete? Yeah.

So think about the difference between a car from 19 80 and a car from tonight.

Yeah. Alright.

No difference, really. Got a bunch of pedals, got a steering wheel. Great. Maybe now it has a few ADAS features, but it's pretty much the same car. Right?

You have no problem getting into a 19 80 car and driving it. You take a programmer today who spent their whole life doing JavaScript, and you put them in an apple to e prompt, and you tell them about the line numbers in basic. But how do I insert something between lines 17 and 18? Wow. But

So you in tool you're putting in the programming languages. So it's just the entirety stack of the tooling. Yeah. So it's not just like the, like, IDs or something like this. Everything.

Yes.

It's Heidi's. The language is the runtimes. It's it's it's everything. And programming is is tool complete. So, like, almost if if if if codecs or or copilot are helping you, that actually probably means that your framework or library is bad, and there's too much boilerplate in it.

Yeah. But don't you think so much programming has boilerplate?

Tiny grad is now 2700 lines. And it can run llama in stable diffusion. And all of this stuff is in 2700 lines. Boiler plate and abstraction interactions and all these things, purchase bad code. Well, Let's talk about good code and bad code.

Yeah.

Because I would say, I don't know, for generic scripts that are right, just offhand. Like, I like, 80 percent of it is written by GPT. Just like quick quick, like, offhand stuff. So not like libraries, not like performing code, not stuff for robotics, and so on. Just quick stuff.

Because your basic so much of programming is doing some some, yeah, boilerplate, but to do so efficiently and quickly because you can't really automate it fully with, like, generic method, like a generic kind of ID type of recommendation or something like this. You do need to have some of the complexity of language models.

Yeah. I guess if I was really writing, like, maybe today if I wrote, like, a lot of, like, data parsing stuff.

Yeah.

I mean, I don't play CTFs any But if I still play CTS a lot of it, like, it's just like you have to write, like, a parser for this data format. Like, I wonder or, like, admin of code. I wonder when the models are gonna start to help with that kind of code, and they may. They may and the models also may help you with speed. Yeah.

My models very fast. But where the models won't, I my programming speed is not at all limited by my typing speed. And in very few cases it is. Yes. If I'm writing some script to just like parse some weird data format, Sure.

My programming speed is limited by my typing speed.

What about looking stuff up? Because that's essentially a more efficient lookup. Right?

You know, when I was at when I was at Twitter, I I tried to use ChatripT to to, like, ask some questions. Like, what's the API for this? Mhmm. And it would just hallucinate. It would just give me completely made up API functions that sounded real.

What do you think that's just a temporary kind of stage? You don't think it'll get better and better and better and this kind of stuff. Because, like, it only hallucinate stuff in in the edge cases. Yes. If your engineering code is actually pretty good.

If you are writing an absolute basic, like, React app with a button, it's not gonna hold this nature. No. There there's kind of ways to fix the hallucination problem. I think Facebook is an interesting paper. It's called Atlas.

And it's actually weird the way that we do language models right now where all of the information is in the white? Mhmm. And human brains don't really like this. It's like a hippocampus and a memory system. So why don't LMs have a memory system?

And there's people working on them. I think future LMs are gonna be like smaller, but are going to run looping on themselves and are going to have retrieval systems. And the thing about using a retrieval system is you can side sources, explicitly,

which is really helpful to integrate the human into the loop of the of the thing because you can go check the sources and you can investigate it. So whenever the thing is hallucinating, you can, like, have the human supervision. That's pushing it towards level 2 kind of That's

gonna kill Google.

Wait. Which part?

When someone makes an LLM that's capable of citing its sources, it will kill Google.

That's citing a source is because that's basically a search engine.

Yeah. That's what people want, miss search engine.

But also Google might be the people that build it.

Maybe.

And put ads on them.

I'd count them out.

Why is that? Why do you think? Who who wins this race? We got who who are the competitors?

Alright.

We got tiny corp. I don't know if that's Yeah. I yeah. I mean, you're a legitimate competitor in that.

I'm not trying to compete on that.

You're not. No. Not as it's gonna accidentally stumble into that competition. Mhmm.

You don't think you

might build the search engines or replace Google search?

When I started a comma, I said, over and over again, I'm going to Win Self Driving Cars. I still believe that. I have never said I'm going to Win Search. With the tiny corp, and I'm never going to say that because I won't.

The night is still young. We don't you don't know how hard is it to win search. In this new route. Like, it's it it feels I mean, 1 of the things that JJPT kinda shows that there could be a few interesting tricks. That really have that create a really compelling product.

Some startups gonna figure it out. I I think I think if you ask me, like, Google is still the number 1 web page. I think by the end of the decade, Google won't be the number 1 web anymore.

So you don't think Google because of the how big the corporation is?

Look, I I would put a lot more money on Mark Zuckerberg.

What is that?

Because Mark Zuckerberg is alive. Like, this is the old Paul Graham essay. Startups are there alive or dead. Google's dead.

Facebook is a live. Facebook is a live. Yeah. It's a live.

Matter. You see what I mean? That's just like like like like Mark Zuckerberg. This is Mark Zuckerberg reading that Paul Grahamassing and being like, I'm gonna show everyone how alive we are. I'm gonna change the name.

So you don't think there's this gutsy pivoting engine that like Google doesn't have that the the the convention and the startup has, like, constantly.

You know what?

Being alive, I guess.

When I was to your Sam Altman, podcast. You talked about the button. Everyone who talks about AI talks about the button, the button to turn it off. Right? Do we have a button to turn off Google?

Is anybody in the world capable of shutting Google down?

What does that mean exactly the company of the of the search engine?

So we shot the search engine down. Kishat the company down. Either?

Can you elaborate on the value of that question?

Does Sundar and Prashay have the authority to turn off Google dot com tomorrow?

Who has the authority? That's a good question. Does anyone? Does anyone? Yeah.

I'm sure.

Are you sure? No. They have the technical power, but do they have the authority? Let's say Saundra Pashay made this his sole mission. Came into Google tomorrow and said, I'm gonna shut Google dot com down.

Yeah. I don't think you'd keep this position too long.

And what is the mechanism by which he wouldn't keep his position?

Well, it's like boards and shares and corporate undermining and, oh my god, our revenue is 0 now.

Okay. So what I mean, what's the case you're making here? So the the capitalist machine prevents you from -- Mhmm. -- having the button.

Yeah. And we'll have a I mean, this is true for the AI's too. Right? There's no turning the AI's off. There's no button.

You can't press it. Now, does Mark Zuckerberg have that button for facebook dot com?

Yes. Probably more.

I think he does. I think he does. And this is exactly what I mean and why I bet on him so much more than I bet on.

I guess you could say Elon has similar stuff.

Oh, Elon has the button. Yeah. You you would. Does Elon can Elon fire the missiles? Can he fire the missiles?

I think some questions are better. You

know who I'm asking. Right? I mean, you know, a rocket at an ICBCL. You have a rocket that can land any Is that an ICBM? Well, yeah, you know, don't ask too many questions.

My god. But the the positive side of the button is that you can innovate aggressively is what you're saying. Which is what's required with training LLM into a search engine?

I would bet on a startup. I bet

Is it so easy? Right?

I bet on something that looks like mid journey, but for search.

Just he's able to size source the loop on itself. I mean, it just feels like 1 model can take off. Yeah. Right? And that nice wrapper, and some of it scam.

It's hard to, like, create a product that just works really nicely, stably.

The other thing that's gonna be cool is there is some aspect of a winner take all effect Right? Like, once someone starts deploying a product that gets a lot of usage, and you see this with OpenAI, they are going to get the data set to train future versions of the model. Yeah. And they are going to be able to right. You know, I was asked to Google and research when I worked there, like, almost 15 years ago now.

How does Google know which image is an apple? And I said, the metadata. And they're like, yeah, that works about half the time. How does Google know? You'll see the raw apples on the front page when you search Apple.

Mhmm.

And I don't know. I didn't come up with the answer. The guys, like, what's what people click on when they search Apple? Mhmm. Like, oh,

Yeah. Yeah. That data is really, really powerful. It's a human supervision. What do you think of are the chances?

What what do you think in general that Llama was open sourced? I just did a conversation with with Mark Zuckerberg, and he's all in on Open Source.

Who would have thought that Marcus Zuckerberg would be the good guy? No, I mean, who

would have thought anything in this world? It's hard to know. But open source to you ultimately is a good thing here.

Undoubtedly, You know, what's ironic about all these AI safety people is they are going to build the exact thing they fear. These, we need to have 1 model that we control and align. This is the only way you end up paperclips. There's no way you end up paper clip if everybody has an AI.

So open sourcing is the way to fight the paper clip maximizer.

Absolutely. It's the only way. You think you're gonna control it? You're not gonna control it.

So the criticism you have for the AI safety folks is that there is belief and a desire for control. Yeah. And that belief and desire for centralized control of dangerous AI systems is not good.

Sam Alpin won't tell you that GPT 4 has 220000000000 parameters and is a 16 way mixture model with 8 sets of weights.

Who did you have to murder to get that information? Alright. I mean, look, but yes.

Everyone at opening eye knows what I just said was true. Right? Now ask the question. Really, you know, it upsets me when I like GP t 2. When opening I came out with GP YouTube and raised a whole fake AI safety thing about that.

I mean, now the model is laughable. Like, they they used AI safety to hype up their company, and it's disgusting.

Or the flip side of that is they used a relatively weak model in retrospect to explore how do we do AI set to correctly, how do we release things, how do we go through the process? I don't I don't know if

I I don't know how

much hype there is an AI safety, honestly.

Oh, there's so much

hype there is an

AI safety, honestly. Oh, there's so much hype. These don't explain there. I don't Maybe Twitter's not real life.

Twitter's not real life. Come on. In terms of hype. I mean, I don't I I think OpenAI has been finding an interesting balance between transparency and putting value on AI safety. You don't think you think just go all out open source.

So do a llama. So do, like, open source this this is a tough question, which is open source both the base, the foundation model, and the fine tune 1. So, like, the the model that can be ultra racist and dangerous and, like, tell you how to build a nuclear weapon.

Oh my god. Have you met humans? Right? Like, half of these AI elements

I haven't met most humans. I this makes this this this allows you to meet every human.

Yeah. I know. But half of these AI alignment problems are just human alignment problems. And that's what's also so scary about the language they use. It's like, It's not the machines you want to align.

It's me.

But here's the thing. It makes it very accessible to ask very questions where the answers have dangerous consequences if you were to act on them.

I mean, yeah. Welcome to the world.

Well, no, for me, there's a lot of friction if I wanna find out how to I don't know, blow up something.

No. There's not a lot of friction that's so easy.

No. Like, what do I search today's being? Or do I what which searched today's

It's like lots of stuff. No.

It feels like I have to

First off. First off. First off. First off. Anyone who's stupid enough to search for how to blow up a building in my neighborhood is not smart enough to build a bomb.

Right?

Are you sure about that?

Yes.

I I feel like I feel like a language model makes it more accessible for that person who's not smart enough to do that.

They're not gonna they're not gonna build a bomb. Trust me. The the the the people the people who are incapable of figuring out how to, like, ask that question a bit more academically and get a real answer from it are not capable of procuring the materials, which are somewhat controlled to build a bomb.

No. I think that all makes it more accessible to people with money with without detecting code know how. Right? To to build like, do you really need to know how to build the bomb to to build a bomb. You can hire people, you can file,

like Or you can hire people to build a you know what? I was asking this question on my stream. Like, can Jeff Bezos hire a hitman? Probably not.

But a language model can probably help you out.

Yeah. And you'll still go to jail. Right? Like, it's not like the language model is gone. Like, the language model.

It's like, you literally just hired someone on fiber. But you you said

okay. Okay. GPT 4 in terms of finding a hitman is like asking fiber how to find

a hit.

I understand. But don't you think

Wicky Hal, you know?

Wicky Hal. But don't you think GPT 5 will be better? Because don't you think that information is out there on the Internet?

I mean, yeah. And I think that if someone is actually serious enough to hire a hitman or build a bomb, they'd also be serious enough to find the information.

I don't think so. I think it makes it more accessible. If you have if you have enough money to buy a hit, man, I think it it decreases the friction. Of how hard is it to find that kind of him. I I honestly think that's the there's a jump in ease and scale of how much harm you can do.

And I don't mean harm with language. I mean harm with the actual violence.

What you're basically saying is like, okay, what's gonna happen is these people who are not intelligent are going to use machines to augment their intelligence and now intelligent people and machines intelligence is scary. Mhmm. Intelligent agents are scary. When I'm in the woods, the scariest animal to meat is human. Mhmm.

Right? No. No. No. No.

There's look, there's, like, nice California humans. Like, I see you're wearing, like, you know, street clothes and nikes, are I fine? Mhmm. But you look like you've been a human. It's been in the woods for a while?

Yeah.

I'm more scared. Are you gonna bear?

That's what they say about the Amazon. You go to the Amazon, it's the human tribes.

Oh, yeah. So intelligence is scary. Right? So to to lack ask this question in generic way. You're like, what if we took everybody who, you know, maybe has ill intention, but is not so intelligent and gave them intelligence?

Right? So we should have intelligence control. Of course, we should only give intelligence to good people, and that is the absolutely horrifying idea

Should you the best defense is actually the best the best defense is to give more intelligence to the to the good guys and intel give intelligence to everybody.

Give intelligence to you know what? It's not even like guns. Right? Like, people say it's about guns. You know, what's what's the best defense against it?

I got a good guy with a gun. Like, I kinda subscribed to that, but I really subscribed to that with intelligence.

Yeah. In a fundamental way, I I agree with you. But there's just feels like so much uncertainty and so much can happen rapidly. That you can lose a lot of control and you can do a lot of damage.

Oh, no. We can lose control. Yes. Thank God. Yeah.

I hope we can I hope they lose control? I want them to lose control more than anything else.

I think when you lose control, you can do a lot of damage, but you can do more damage when you centralized and hold on to control is the point of

centralized and held control is tyranny. Right? I will always I don't like anarchy either, but I'll always take anarchy over tyranny. Anarchy, you have a chance.

This human civilization got going on is quite interesting. I mean, I agree with you. So do you open source is the way forward here. So you admire what Facebook is doing here or what Matt is doing with the release of

a fellow? I lost I lost 80000 dollars last year investing in Meta. And when they released LAMA, I'm like, yeah, whatever, man. That was worth it.

That's worth it. Do you think Google and OpenAI with Microsoft will match? What what what matter is doing, you know?

So if I were a researcher, Why would you wanna work at opening? Like, you know, you're you're just you're on the bad team. Like, I mean it. Like, you're on the bad team who can't even say that GPT 4 has 2 20000000000 parameters.

So close source to use the bad team.

Not only close source, I'm not saying you need to make your model way to open. Mhmm. I'm not saying that. I totally understand. We're keeping our model weights closed because that's our product.

Right? That's fine. I'm saying, like, because if AI safety reasons, we can't tell you the number of billions of parameters in the model. That's just the bad guys.

Just because you're mocking AI safety doesn't mean it's not real.

Oh. Of course.

Is it possible that these things can really do a lot of damage that we don't know.

Oh my god. Yes. Intelligence is so dangerous, be it human intelligence or machine intelligence? Intelligence is dangerous.

To put machine intelligence is so much easier to deploy at scale, like, rapidly. Like, what? Okay. If you have human like bots on Twitter -- Right. -- and you have, like, a thousand of them create a whole narrative.

Like, you can manipulate millions of people.

But you mean like the intelligence agencies in America are doing right now?

Yeah. But they're not doing it that that well. It feels like you can do a lot.

They're doing it pretty well. Well, I can get doing a pretty good job.

I I suspect they're not nearly as good as a bunch of GPT fueled bots could be?

Well, I mean, of course, they're looking into the latest technologies for control of people, of course.

But I I think there's George Cross type care that can do a better job than the entire idea of the you don't think so.

No. And I'll tell you why the George Hotz character can't. And I thought about this a lot with hacking. Right? Like, I can find exploits in web browsers.

I probably still can. I mean, I was better. I don't know if it's 24, but The thing that I lack is the ability to slowly and steadily deploy them over 5 years. And this is what intelligence agencies are very good at. Right?

Intelligence agencies don't have the most sophisticated technology. They just have

endurance endurance. In yeah. The financial backing and the infrastructure for the Endurance.

So the more we can be centralized power. Like, you could make an argument by the way that nobody should have these things. And I would defend that argument. I would I would like you saying, look, LLMs and AI and machine intelligence can cause a lot of harm, so nobody should have it. And I will respect someone philosophically with that position.

Just like I will respect someone philosophically with the position that nobody should have guns. Right? But I will not respect philosophically which with with only the trusted authorities should have access to this. Yeah. Who are the trusted authorities?

You know what? I'm not worried about alignment between AI company and their machines. I'm worried about alignment between me and AI company.

What do you think oh, the Azerika Kalske would say to you? This is really against open source.

I know. And I thought about this. I thought about this. And I think this comes down to a repeated misunderstanding of political power by the rationalists.

Interesting.

I think that Elias Yudkowski is scared of these things, and I am scared of these things too. Everyone should be scared of these things. These things are scary. But now you ask about the 2 possible futures. 1 where a small trusted centralized group of people has them and the other where everyone has them.

And I am much less scared of the second future than the first.

Well, there's a small trusted group of people that have control of our nuclear weapons.

There's a difference. Again, a nuclear weapon cannot be deployed tactically and a nuclear weapon is not a defense against a nuclear weapon. Except maybe in some philosophical mind game kind of way.

But AI is different different how exactly?

Okay. Let's say the Intelligence agency deploys a million bots on Twitter or a thousand bots on Twitter to try to convince me of a point. Mhmm. Imagine I had a powerful AI running on my computer saying, okay. Nice eye up.

Nice eye up. Nice eye up. Okay. Here's a thigh up. I filtered it out for you.

Yeah. I I mean, so you have a fundamental hope for that, for the for the defensive side up.

I'm not even like, I don't even mean these things and, like, truly horrible. Ways. I mean these things in straight up like ad locker. Right?

Yeah.

Straighten ad locker. Right? I don't wanna ads. Yeah. But they are always finding, you know, imagine I had an AI that could just block all the ads for me.

So you believe in the the power of the people to always create a now blocker. Yeah. I mean, III kinda share that belief. I have that that's the 1 of the deepest optimism I have is just, like, there's a lot of good guys So to give you know, you shouldn't handpick them. Just throw out powerful technology out there.

And the good guys will outnumber and outpower the bad guys.

Yeah. I'm not even gonna say there's a lot of good guys. I'm saying that good outnumber is bad. Right? Go down numbers bad

in skill and performance.

Yeah. Definitely in skill and performance. Probably just a number too. Probably just in general. I mean, you know, if you believe philosophically in democracy, Leave that that good out numbers.

Bad. Yeah. And, like, the only if you give it to a small number of people, There's a chance you gave it to good people, but there's also a chance you gave it to bad people. If you give it to everybody, well, if good out, number's bad, then you definitely gave it to more good people than bad.

That's really interesting. So that's on the safety grounds. But then also, of course, there's other motivations like, you don't wanna give away your secret sauce.

Well, that's what I mean. I mean, I I look I respect capitalism. I don't think that I think that it would be polite for you to make model architectures open source and fundamental breakthroughs open source. Don't think you have to make way to open source.

You know what's interesting is that, like, there's so many possible trajectories in human history where you could have the next Google be open source. So for example, I don't know if that connection is accurate, but, you know, Wikipedia made a lot of interesting decisions not to put ads. Like, Wikipedia is basically open source. You could think of it that way.

Yeah.

And, like, that's 1 of the main websites on the Internet.

And,

like, it didn't have to be that way. It could have been, like, Google could have created Wikipedia, put ads on it. You could probably run amazing ads now in Wikipedia. You wouldn't have to keep asking for money, but it's interesting. Right?

So Llama, Open Source Llama, Derivatives of Open Source Llama might win the Internet?

I sure hope so. I hope to see another era. You know, the kids today don't know how good the Internet used to be. And I don't think this is just come on. Like, everyone's nostalgic for their past, but I actually think the Internet before small groups of weaponized corporate and government interests took it over was a beautiful place.

You know, those small small number of companies have created some sexy products. But you're saying, overall, in the long arc of history -- Mhmm. -- the centralization of power they have, like, suffocated the human spirit scale.

Here's a question to ask about those beautiful sexy products. Imagine 2000 Google to 20 10 Google. Right? Mhmm. A lot changed.

We got maps. We got Gmail. Mhmm.

We lost a lot of products too, I think.

From yeah. I mean somewhere probably we've got Chrome. Right? And now let's go from 20 10. We got Android.

Now let's go from 20 10 to 20 20. What does Google have? What does Search Engine, Maps, Mail, Android, and Chrome? I see. The the Internet was this You know, I was times first of the year in 2006?

Yes.

I love this.

Yeah. It's you. Was times first of the year in 2006? Alright. Like like, that's You know, so quickly did people forget?

And I think some of it's social media. I think some of it I I hope, look, I hope that I I don't it's possible that some very sinister things happen. I don't I don't know. I think it might just be like the effects of social media. But something happened in the last 20 years.

Okay.

No. Oh, okay. So you're you're just being an old man. I was worried about that. Think there's always it goes it's the cycle thing.

It's ups and downs, and I think people rediscover the power of distributed of decentralized. Yeah. I mean, that's kinda like what the the whole, like, cryptocurrency is trying, like, that that I think crypto is just carrying the flame of that spirit of, like, self should be designed

it's just such a shame that they all got rich. You know? Yeah. If you took all the money out of crypto, it would have been a beautiful place.

Yeah.

But, no, I mean, these people, you know, they they sucked all the value out of it and took it.

Yeah. Money kinda corrupts the minds somehow. It becomes a drug.

He interrupted all of crypto. You had coins worth billions of dollars that had 0 use.

Yeah. You still have hope for crypto?

Sure. I have hope for the ideas. I really do. Mhmm. Yeah.

I mean, you know, I want the US dollar to collapse.

I do. George Watts. Well, let me sort of on on the a ASAT. Do you think there's some interesting questions there, though, to solve the open source community in this case? So, like, alignment, for example, or the control problem.

Like, if you really have super powerful, you said it's scary. Oh, yeah. What do we do with it? So not not control, not self control, but, like, if you were then you're gonna see some guy or gal release a super powerful language model open source. And here you are George Cost thinking, holy shit.

Okay. What ideas do I have? To combat this thing. So what ideas would you have?

I am so much not worried about the machine independently doing harm. That's what some of these AI safety people seem to think. They somehow seem to think that the machine, like, independently, is gonna rebel against its creator.

So you don't think you'll find autonomy?

No. This is sci fi B movie garbage.

Okay. What if the thing writes code? Basically writes viruses.

If the thing might write viruses, it's because the human told it to write viruses.

Yeah. But there's some things you can't, like, put back in the box. That's that's kind of the whole point -- What? -- is it kind of spreads. Give it access to the Internet -- Mhmm.

-- it spreads, installs itself. Modifies your shit.

BBBBB plot sci fi

Not real unless I'm trying to work. I'm trying to get better in my plot writing.

The thing the thing that worries me, I mean, we have a real danger to discuss and that is bad humans using the thing to do whatever bad unaligned AI thing you want.

But this goes to the your previous concern that who gets defined, who's a good human, who's a bad human.

Nobody does. We give it to everybody. And if you do anything besides, give it to everybody, trust me, the bad humans will get it. And it's natural gas power. It's always the bad humans who get power.

Okay.

Power. And power turns even slightly good humans to bed. Sure. That's the intuition you have. I don't know.

I don't think everyone. I don't think everyone. I just think that like Here here's the saying that I put in 1 of my blog posts. When I was in the hacking world, I found 95 percent of people to be good and 5 percent of people to be bad. Just who I personally judged as good people and bad people.

Mhmm. Like, they believed about, like, you know, good things for the world. They wanted, like, flourishing, and they wanted, you know, growth that they wanted. Things like considered good. Right?

Mhmm. I came into the business world with comma, and I found the exact opposite. I found 5 percent of people go ahead and 95 percent of people bet. I found a world that promotes psychopathy.

I wonder what that means. I wonder if that care like, I wonder if that's anecdotal or if it if that's true to that, there's something about capitalism. Wow. At the core that promotes the people that run capitalism that promotes psychopathy.

That's saying, may, of course, be my own biases. Right? That may be my own bias is that these people are a lot more aligned with me than these other people. Right? Yeah.

So, you know, I I can certainly recognize that. But, you know, in general, I mean, this is like like the common sense Maxim, which is the people who end up getting power are never the ones you want with.

But do you have a concern of superintelligent AGI? Open sourced. And then what do you do with that? I'm not saying, control it. It's open source.

What do we do with this human species?

If that's not up to me, I mean, you know, like, I'm not a central planner. Well,

not a central planet, but you'll probably tweet as a few days left to live for the US pieces.

I have my ideas of what to do with it, and everyone else has their ideas of what to do with it. They make the best ideas

But at this point, do you brainstorm, like because it's not regulation. It could be decentralized regulation where people agree that this is just like We create tools that make it more difficult for you to maybe make it more difficult for code to spread, you know, antivirus software, this kind of thing. But you're

saying that you should build AI firewalls. That sounds good. You should definitely be running an AI firewall. Yeah. Right.

You should be running an AI firewall to your mind. Right? You're constantly under, you know,

such an interesting idea.

In full wars, man, like,

I I don't know if you've been sarcastic. No. I'm dead serious. But certain things that's power to that is, like, how do I protect my mind from influence of human like or super human intelligent bots?

I it's not being I would pay so much money for that product. I would pay so much money for that product. I would you know how much money I'm paying just for a spam filter? That works.

Well, on Twitter, sometimes I would like to have a a protection mechanism for my mind from the outrage mobs.

Yeah.

Because they feel like bot like behavior. It's like -- Yeah. -- it's a large number of people that will just grab a viral narrative and attack anyone else to believe otherwise. And it's like,

whenever someone's telling me some story from the news, I'm always like, I don't wanna hear it CIA op pro. It's the CIA op pro. Like, it doesn't matter if that's true or not. It's just trying to influence your mind. You're repeating an ad to me.

Like the viral mobs, is it like the yeah. There.

No. To me, a defense against those those mobs is just getting multiple perspectives always from from sources that make you feel kinda like you're getting smarter and just actually just basically feels good. Like, a good documentary just feels good. Something feels good about it. It's well done.

It's like, oh, okay. I never thought of it this way. Just feels good. Sometimes the outrage mobs, even if they have a good point behind it, when they're like mocking and derrusive and just aggressive, you're with us or against us, this this fucking This is why I delete my tweets. Yeah.

Why'd you do that? I was, you know, I was I missed your tweets.

You know what it is? The algorithm promotes toxicity. Yep. And like, you know, I think Elon has a much better chance of fixing it than the previous regime. Yeah.

But to solve this problem, to solve like, to build a social network that is actually not toxic without moderation.

Mhmm. Like, not to stick but carrots. So, like, when people look for goodness, So make it catalyze the process of connecting cool people and being cool to each other.

Yeah.

Without ever sensor.

Without ever censoring. And and and, like, Scott Alexander has a blog post that, like, were you talking about, like, moderation? It's not censorship. Right? Like, all moderation you wanna put on Twitter Right?

Like, you could totally make this moderation like just a You don't have to block it for everybody. You can just have like a filter button. Right? That people can turn off. If they were, like, say, search for Twitter.

Right? Like, someone could just turn that off. Right? So, like but then you would, like, take this idea to an extreme. Right?

Well, the network should just show you this is a couchsurfing CEO thing. Right? If it shows you right now, these algorithms are designed to maximize engagement. Well, turns out outrage maximizes engagement. Quirk of human quirk of the human mind.

Right? Just this. I fall forward, everyone falls forward. So, yeah, you gotta figure out how to maximize for something other than engagement.

And I I actually believe that you can make money with that too. So it's not I don't think engagement is the only way to make money.

I actually think it's incredible that we're starting to see I think, again, you're only doing so much stuff right with Twitter, like, charging people money. As soon as you charge people money, they're no longer the product. They're the customer. And then they can start building something that's good for the customer and not good for the other customer, which is the ad agencies.

Asn't asn't picked up his team.

I pay for Twitter. Doesn't even get me anything? It's my donation to this new business model hopefully working out.

Sure. But, you know, you for this business model to work, it's like most people should be signed up to Twitter. And so the way it was there was something but perhaps not compelling, there's something like this to to people.

Think you need most people at all. I think that I why do I need most people? Right? Don't make an 8000 person company, make a 50 person company.

Well, so speaking of which, he worked at Twitter for a bit, that did. As an intern.

Mhmm.

The world's greatest intern.

Yeah.

Alright.

There's been better.

That's been better. Tell me about your time at Twitter. How did it come about? And what did you did you learn from the experience?

So I deleted my first Twitter in 20 10. I had over a hundred thousand followers back when that actually meant something. And I just saw, you know, My coworker summarized it well. He's like, whenever I see someone's Twitter page, I either think the same of them or less of them. I never think more of that.

Yeah. Right? Like like, you know, I don't I don't wanna mention any names, but like some people who like, you know, maybe you would like read their books and you would respect them. You see them on Twitter and you're like, Okay, dude.

Yeah. But there are some people with the same. You know who I respect a lot. Are people that just post really good technical stuff. Yeah.

And I guess, I don't know. I think I respect them more for it, because you you realize, oh, this wasn't there's, like, so much depth to to this person, to their technical understanding of so many different topics.

Okay.

So I try to follow people. I try to consume stuff this technical machine learning content.

There's probably a few of those people. And the problem is inherently what the algorithm reward. Right? And people think about these algorithms. People think that they are terrible awful things.

And, you know, I love that Elon open sourced it. Because, I mean, what it does is actually pretty obvious. It just predicts what you are likely to retweet and like -- Mhmm.

--

and linger on. So all these algorithms do is, which they talk to us. So all these recommendation ideas do. And it turns out that the thing that you are most likely to interact with is outreach. And that's quirk of the human condition.

I mean and there's different flavors of outrage. It doesn't have to be it could be mockery, You could be outraged. The topic of outrage could be different. It could be an idea. It could be a person.

It could be and and maybe there's a better word than outrage. It could be drama.

Sure.

All these kind of stuff. Yeah. But doesn't feel like when you consume it, it's a constructive thing for the individuals that consume it in the long term.

Yeah. So my time there, I absolutely couldn't believe, you know, I got crazy amount of hate. You know, on Twitter for working at Twitter. It seems like people associated with this, I think maybe you were exposed to some of this.

To connection to Elon, or is it working on Twitter?

Twitter and Elon, like, the whole

There's Elon's gotten a bit spicy during that time. A bit political, a bit?

Yeah. Yeah. You know, I I remember 1 of my tweets. It was never go full of Republican, and Elon liked it. You know what I mean?

Oh, boy. Yeah. I mean, there's a roller coaster of that, but being political on Twitter -- Yeah. -- boy.

Yeah.

And also being just attacking anybody on Twitter, it comes back at you harder. And if it's political and attacks?

Sure. Sure. Absolutely.

And then letting sort of deplatform people back on even adds more fun to the to the to the beautiful chaos.

I was hoping. And, like, I remember when Elon talked about buying Twitter, like, 6 months earlier, he was talking about, like, a principled commitment to free speech. And I'm a big believer in fan of that. I would love to see an actual principled commitment to free speech. Of course, this isn't quite what happened.

Instead of the oligarchy deciding what to ban, you had a monarchy deciding what to ban. Right? Instead of, you know, all the Twitter files, shadow and, really, their geography just decides what? Cloth masks are ineffective against COVID. That's a true statement.

Every doctor in 20 19 knew it. And now I'm banned on Twitter for saying it. Interesting. Oligarchy. So now you have a monarchy and, you know, he bands things he doesn't like.

So, you know, it's just it's just different it's different power and, like, you know, maybe I Maybe I align more with him than with the oligarchy.

But it's not a piece

of speech absolutely.

But I I feel like being a piece of speech absolutely to the social network acquire is you to also have tools for the individuals to control what they consume easier. Like, not sensor.

You know what I mean?

But just to control, like, oh, I like to see more cats and less politics.

And this isn't even this isn't even remotely controversial. This is just saying you want to give paying customers for a product what they want.

Yeah. Right? They're not through the process of censorship, but through the process of like

Well, it's individual. Right? It's individualized transparent censorship, which is honestly what I want. What is an ad blocker? It's individualized transparent censorship.

Right?

Yeah. But censorship was a strong word. And people are very sensitive too.

I know. But, you know, III just use words to describe what they functionally are. And what is an ad blocker? It's just censorship.

You know, when I look at it, you know, I know. Sensory. I'm looking at you. I'm censoring everything else out when I'm when my mind is focused on you. That's you can use the word transcript that way.

But usually, when people get very sensitive about the censorship thing. I I think when you have when anyone is allowed to say anything, you should probably have tools that maximize the quality of the experience for individuals. So, like, you know, for me, like, what I really value, boy, would be amazing to somehow figure out how to do that. I love disagreement and debate and people who disagree with each other disagree with me, especially in the space of ideas, but the high quality ones. So not duration.

Right? Maslow's hierarchy of argument. I think it's a real word for it.

Probably. Yeah. There's just a way of talking this snarky and so on that somehow is gets people on Twitter and they get excited and so on.

We have, like, ad hoc and him refuting the central point. I've, like, seen this as an actual pyramid itself.

Yeah. It's yeah. And it's it's like all of it all the wrong stuff is attractive to people.

I mean, we can just train a classifier to absolutely say what level of Maslow's hierarchy of argument are you at Yeah. If it's ad hominem, like, okay. Cool. I turned on the no ad hominem filter.

I wonder if there's a social network that will allow you to have that kind of filter.

Yeah. So here's a problem with that. It's not going to win in a free market.

Yeah.

What wins in a free market is all television today is reality television because it's engaging. Right. If if engaging is what wins in a free market. Right? So it becomes hard to keep these other more nuanced values.

Well, okay. So that's the experience of being on Twitter. But then you got a chance to also together with other engineers and with Elon sort of look brainstorm when you step into a code base.

Mhmm.

It's been around for a long time. You know, there's other social networks, you know, Facebook. This is old code basis. And step in and see, okay, how do we make with a fresh mind progress on this code base? Like, what what what did you learn about software engineering, about programming from just experience in that.

So my technical recommendation to Elon and I said this on the Twitter space is afterward. I said this many times during my brief internship was that you need refactors before features. This code base was And look, I've worked at Google. I've worked at Facebook. Facebook has the best code.

Then Google, then Twitter. And you know what? You can know this because look at the machine learning frameworks. Right? Facebook released PyTorch.

Google released TensorFlow and Twitter released Okay. So, you know, it is.

It's a proxy, but yeah. There the the Google Clubbase is quite interesting. There's a lot of really good software engineers there, but the Clubbase is very large.

The code base was good in 20 in 2005. Right? It looks like 2005 errors.

There's so many thought of so many teams. Right? It's very difficult to I feel like Twitter does less, like, obviously, much less than Google. In terms of, like, the set of features. Right?

So, like, it's I can imagine the number of software engineers that could recreate Twitter is much smaller than to recreate Google.

Yeah. I still believe and the amount of hate I got for saying this that 50 people could build and maintain Twitter.

Pretty what's the nature of the hate comfortably that you don't know what you're talking about?

You know what it is? And it's the same This is my summary of like the hate I get on Hacker News. It's like, when I say I'm going to do something, they have to believe that it's impossible. Yeah. Because if doing things was possible, they'd have to do some soul searching and ask the question, why didn't they do anything?

So when you say

And I do say that's where the hate comes.

When you say, well, there's the core truth. That. Yeah. So when you say I'm gonna solve self driving, people go, like, what are your credentials? What the hell are you talking about?

What do you this is extremely problem. Of course, you're noob that doesn't understand the problem deeply. I mean, that that was the same nature of hate that probably you long ago when you first talked about Adam was driving. But, you know, there there's pros and cons to that because, like, you know, there is experts in this world.

No. But the the mockers aren't experts. The market the the the people who are mocking are not experts with carefully reasoned arguments about why you need 8000 people to run a BERT app. There. But the people are gonna lose their jobs.

Well, that but also just the software engineers, the proper career says, no, it's a lot more complicated than you realize. But maybe doesn't need to be so complicated.

You know? Some people in the world like to create complexity. Some people in the world thrive under complexity like lawyers. Right? Lawyers want the world to be more complex because you need more lawyers, you need more legal hours.

Right? I think that's another. If there's 2 great evils in the world, it's centralization and complexity.

Yeah. And it the 1 of the sort of hidden side effects of software engineering is like, finding pleasure and complexity. I I mean, I don't remember just taking all the software engineering courses and doing programming and and this is just coming up in this object oriented programming kind of idea. You don't like, not often do people tell you, like, do the simplest possible thing. Like like a a professor, a teacher who's not gonna get in front, like, this is the simplest way to do it.

They'll say, like, this is direct there's the right way. And the right way, at least for a long time, you know, especially I came up with, like, Java. Right? Like, is is is there so much boilerplate, so much, like, so many classes, so many, like, designs and architectures and so on, like planning for features far into the future -- Mhmm. -- and planning poorly and all this kind of stuff.

And then there's this, like, code base. It follows you along. It puts pressure on you. And nobody knows what, like, parts, different parts do, what slows everything down. It's a kind of bureaucracy that's instilled in the code as a result of that.

But then you feel like, oh, well, I follow good software engineering practices. It's it's an interesting trade off because then you look at, like, the get oness of, like, pearl in the old, like, how quickly you could just write a couple lines and you could get stuff done. That trade off is interesting, or bash, or whatever, these kind of get all things you can do in Linux.

1 of my favorite things to look at today is how much do you trust your tests. Right? We've put a ton of effort in comma and I put a ton of effort in tiny grad into making sure if you change the code and the tests pass, that you didn't break the

code. Yeah.

Now, obviously, it's not always true. But the closer that is to true. The more you trust your tests, the more you're like, oh, I got a pull request and the tests pass. I feel okay to merge that, the faster you can make progress.

He always programming with tests in mind, developing tests -- Yeah. -- with with that in mind that if it passes, it should be good.

And Twitter had

Not that. So it was impossible

to make progress in the cobase.

What other stuff can you say about the cobase that made it difficult? And what are some interesting sort of quirks? Broadly speaking, so that compared to just your experience with comma and everywhere else?

The real thing that I I spoke to a bunch of, you know, like like like the individual contributors

of Twitter.

And I I just had stats. I'm like, okay. So, like, What's wrong with this place? Why does this code look like this? And they explained to me what Twitter's promotion system was.

The way that you got promoted to Twitter was you wrote a library that a lot of people used. Right? So some guy wrote an NGINX replacement for Twitter. Why does Twitter need an NGINX replacement? What was wrong with NGINX?

Well, you see you're not gonna get promoted if you use NGINX. But if you write a replacement and lots of people start using it as the Twitter front end for their product, then you're gonna get promoted. Right?

Someone should think there's, like, from an individual perspective, how do you incentivize? How do you create the kind of incentives that will reach a lead to a great code code base? What's okay. What's the answer to that?

So what I do at comma and at and then, you know, at Tiny Corp is you have to explain it to me. You have to explain to me what's code does. Right? And if I can sit there and come up with a simpler way to do it, you have to rewrite it. You have to agree with me about the simpler way.

You know, obviously, we can have a conversation about this. It's not a it's not dictatorial. But if you're like, wow. Wait. That actually is way simpler.

Like like, the simplicity is important.

Right? But that requires people that overlook the code at the at the highest levels to be like, okay.

It requires technical leadership each

yeah, tech technical leadership. So managers or whatever should have to have technical savvy, deep technical savvy.

Manager should be better programmers than the people who they manage.

Yeah. And that's not how always obvious the attributable to create, especially large companies, managers get soft.

And, like, you know, and this is just I've instilled this culture at comma, and comma has better programs than me who work there. Mhmm. But, you know, again, I'm, like, the, you know, the old guy from goodwill hunting. It's like, look man, you know, I might not be as good as you, but I can see the difference between you and you. Right?

Yeah. This is what you need. This is what you need at the top, or you don't necessarily need the manager to be Absolute best. I shouldn't say that. But like, they need to be able to recognize scale.

Yeah. And have good intuition. Intuition that's laden with wisdom. From all the battles of trying to reduce complexity in code basis.

You know, I took a I took a political approach at Kama too that I think is pretty interesting. I think Elon takes the same political approach. You know, Google had no politics. And what ended up happening is the absolute worst kind of politics took over. Comma has an extreme amount of politics and they're all mine and no dissidents is tolerated.

So it's a dictatorship.

Yep. It's an absolute dictatorship. Right? Elon does the same thing. Now, the thing about my dictatorship is here are my values.

Yeah. It's transparent.

It's transparent. It's a transparent dictatorship. Right? And you can choose to update or, you know, you get free exit. Right?

That's a beauty of companies. If you don't like the dictatorship, click.

So you mentioned rewrite before or refactor before features.

Mhmm.

If you were to refactor the Twitter code base, what would that look like? And maybe also comment on how difficult is it to refactor?

The main thing I would do is, first of all, identify the pieces and then put tests in between the pieces. Right? So there's all these different Twitter as a micro surface architecture. Early different microservices. And the thing that I was working on there, look, like, you know, George didn't know any JavaScript.

He asked how to fix search blah blah blah blah blah. Look man, like The thing is, like, I'm just, you know, I'm upset at the way that that this whole thing was portrayed because it wasn't like it wasn't like taking my people, like, honestly. It wasn't like by it was taken by people who started out with a bad faith assumption. Yeah. And, yeah, I mean, look, I can't like.

And you as a programmer just being transparent out there, actually, having like, fun. And, like, this is what program it should be about.

And, like, I love that Elon gave me this opportunity. Yeah. Like, really, it it does. And, like, you know, you can't remember my my the the the day I quit, it came on my Twitter spaces afterward, and we had a conversation. Like, I just I respect that so much.

Yeah. And it's also inspiring to just engineers and programmers and just It's cool. It should be fun. The be the people that were hating on it is like, oh, man.

It was fun. It was fun. It was stressful. But I felt like, you know, was that like a cool, like, point in history and, like, I hope I was useful and probably kind of wasn't. But, like,

maybe Oh, you also were 1 of the people that kind of made a strong case to refactor. Yeah. And that that's a really interesting thing to raise. Like, maybe that is the right you know, the timing of that is really interesting. If you look at just the development of auto pilot, you know, going from Mobileye to just, like, more if you look at the history of Simeon was driving in Tesla, is is more and more like you could say refactoring or or starting from scratch, redeveloping from scratch.

3 factoring all the way down.

And like and the question is, like, can you do that sooner? Can you maintain product profitability And, like, what's the what's the right time to do it? How do you do it? You know, on any 1 day, it's like, you don't wanna pull off the Band A. It's like it's Like, everything works.

It's just, like, little fixed here and there, but maybe started from scratch.

This is the main philosophy of tiny grad. You have never refactored enough. Your code can get smaller, your code can get simpler, your ideas can be more elegant.

But would you consider you know, say you are like running Twitter development teams, engineering teams. Would you go as far as, like, different programming language? Just go that far.

I mean, the first thing that I would do is build tests. The first thing I would do is get a CI to where people can trust to make changes.

So that if you

keep touched any code, I would actually say no 1 touches any code. The first thing we do is we test this code base. I mean, this is classic. This is how you approach a legacy code base. This is like what any how do we approach a legacy code base book we'll tell you?

So

And then you hope that there's modules that can live on for a while, and then you add new ones, maybe in a different language or before we sign

the new ones, we replace old ones.

Yeah. Yeah. Meaning, like, replace old ones with something simpler.

We we look at this, like, this thing that's a hundred thousand lines and we're like, okay, maybe this did even make sense in 20 10, but now we can replace this with an open source thing. Right? Yeah. And, you know, we look at this here. Here's another 50000 line Well, actually, you know, we can replace this with 300 lines of go.

Mhmm. And you know what? I trust that the go actually replaces this thing because all the tests still pass. So step 1 is testing.

Yeah.

And then step 2 is, like, the programming languages that I have to thought. Right? You know, let a whole lot of people compete. Be like, okay. Who wants to rewrite a module?

Whatever language you wanna write it in, just the tests have to pass? And if you figure out how to make the test pass, but break the site, that's We gotta go back to step 1. Step 1 is get tests that you trust in order to make changes in the code pass.

I wonder how harder it is too, because I'm I'm with you on on testing and everything. I have from tests to, like, a search to everything with code is just covered in this because it should be very easy to make rapid changes and no, that's not gonna break everything. And that's the way to do it. But I I wonder how difficult is it to integrate tests into a code base that doesn't have many of those.

So I'll I'll I'll tell you what my plan was at Twitter. It's actually similar to something we use a comma. So a comma we have this thing called process replay. Mhmm. We have a bunch of routes that'll be run through.

So commas and microservices architecture too. With microservices in the driving, we have 1 for the cameras, 1 for the sensor, 1 for the planner, 1 for the model. And we have an API, which the microsurfaces talk to each other with. We use this custom thing called serial, which uses ZMQ. Twitter uses thrift, and then it uses this thing called Fanegal, which is a scala RPC back end, but this doesn't really matter.

The thrift and finagled layer was a great place I thought to write tests.

Mhmm. Right?

To start building something that looks like process replay. So Twitter had some stuff that looked kind of like this. But it wasn't offline. It was only online. So you could ship, like, a modified version of it and then you could redirect some of the traffic to your modified version and dip those to.

Mhmm. But it was all online. Like, there was no, like, CI in the traditional sense. I mean, there was some, but, like, it was not full coverage.

She can't run all of Twitter offline to test something.

Well, then this was another problem. You can't run all of Twitter.

Right? Period. Twitter. 1 person can't.

Twitter runs in 3 data centers, and that's it. Yeah. There's no other place you can run Twitter, which is like George, you don't understand? This is modern software development. No.

This is bullshit. Like, do I get it run on my laptop? What do you do? Twitter can yeah, okay. Well, I'm I'm not saying you're gonna download the whole database to your laptop, but I'm saying all the middleware and the front end should run on my laptop.

Right?

That sounds really compelling. Yeah. But can that be achieved by a code base that grows over the years? I mean, the 3 data centers didn't have to be. Right?

Because it's they're totally different, like, designs.

The problem is more, like like, why did the code base have to draw what new functionality has been added to compensate for the the lines of code that are there.

1 of the ways to explain is that the incentive for software developers to move up in the companies to add code, to add especially laws.

You don't want the incentive for politicians to move up in the political structures, to add laws, Same problem.

Yeah. Yeah. If the flipside is to simplify, simplify, simplify. I mean,

You know what? This is something that I do differently from from Elon with with Kauma about self driving cars. You know, I hear the new version's gonna come out, and the new version is not gonna be better, but at first, and it's gonna require a ton of refactors. And I say, okay. Take as long as you need.

Like, you convinced me this architecture is better? Okay. We have to move to it. Even if it's not gonna make the product better tomorrow, the top priority is making is getting the architecture. Right?

So what do you think about sort of a a thing where the product is online? So how I guess, would you do a refact if you ran engineering and Twitter, would you just do a refactor? How long would it take? What would that mean for the running of the of the actual service?

You know? And I'm not the right person to run Twitter. I'm just not. And that's the problem. Like like, I don't really know.

I don't really know if that's, you know, a common thing that I thought a lot while I was there. Was whenever I thought something that was different to what Elon thought. I have to run something in the back of my head reminding myself that Elon is the richest man in the world. And in general, his ideas are better than mine. Mhmm.

Now, there's a few things I think I do understand and know more about. But, like, in general, I don't qualify to run Twitter. No. Or as soon as I qualify, but, like, I don't think I'd be that good at it. I don't think I'd be good at it.

I don't think I'd really be good at running an engineering organization at scale. I think I could lead a very good refactor of Twitter. And it would take, like, 6 months to a year. And the results to show at the end of it would be feature development in general. It takes 10 x less time.

10 x less man hours. That's what I think I could actually do. Do I think that it's the right decision for the business above my pay grade?

Yeah. But a lot of these kinds of decisions are above everybody's pay grade.

I don't wanna be a manager. I don't wanna do that. I just like like you if you really forced me to, yeah, it would make me maybe make me upset. If I had to make those decisions, I don't I don't wanna

Yeah. But a refactor is so compelling. If this is to become something much bigger than what Twitter was, is it feels like a refactors has to be coming at some point.

George George, junior software engineer, every junior software engineer to come in and refact the ball COVID. Okay. Like, that's, like, your opinion, man.

Yeah. It doesn't, you know, sometimes they're right.

Well, like, whether they're right or not, it's definitely not for that reason. Right? It's definitely not a question of engineering prowess. It is a question of maybe what the priorities are for the company. And I did get more intelligent, like, feedback from people, I think, in good faith, like saying that.

But from actually, from Milan, And, like, you know, from from from Milan, sort of, like, like, people were like, well, you know, a stop the world refactor might be great for engineering, but, you know, we have business to run. And, hey, above my pay grade.

Would you think about Elon as an engineering leader having to experience him in the most chaotic of spaces, I would say.

My respect for him is unchanged. And I did have to think a lot more deeply about some of the decisions he's forced to make.

About the tensions within those, the trade offs within those decisions?

About like a whole, like, like, matrix coming at him. I think that's Andrew Kate's word for it. Sorry to borrow it.

Also, bigger than engineering. Just everything.

Yeah. Like like the war on the woke. Yeah. Like, it yeah. It's just it's just man and, like, he doesn't have to do this, you know?

He doesn't have to. He could go, like, Parag and go chill at the 4 seasons of Maui, you know? But See, 1 person I respect and 1 person I don't.

So his heart is in the right place, fighting in this case for this ideal of the freedom of expression.

But I wouldn't define the ideal so simply. I think you can define the ideal no more than just saying, Elan's idea of a good world. The freedom of expression is,

but to you, it's still the downsides of that is the monarchy.

Yeah. I mean, monarchy has problems. Right? But, I mean, would I trade right now the mona or the current oligarchy? Which runs America for the monarchy?

Yeah. I would. Sure. For the Elon Monarchy? Yeah.

You know why? Because power would cost 1 cent a kilowatt hour. 10 to a cent a kilowatt hour?

What do you mean?

Right now, I pay about 20 cents a kilowatt hour for electricity in San Diego. That's, like, the same price you paid in 19 80? What the hell?

So you you would see a lot of innovation -- Yeah. -- with Elon.

Maybe you'd have maybe have some hyper loops.

Yeah.

Right? And I'm willing to make that trade off. Right? I'm willing to make and this is why, you know, people think that, like, dictators take power through some like through some untoward mechanism. Sometimes they do, but usually it's because the people want them.

And the downsides of a dictatorship, I feel like we've gotten to a point now with the oligarchy where. Yeah, I would prefer the dictator.

What'd you think about Skyler as a programming language?

I liked it more than I thought. I did the tutorials. The guy's very new to it. Like, it would take me 6 months to be able to write, like, good scala.

I mean, what did you learn about learning a new programming language from that?

I love I love doing, like, new programming guide tutorials and doing them. I did all this for rust. It keeps some of its upsetting JBM roots. But it is a much nicer. In fact, I almost don't know why cotton took off and not scala.

Mhmm. I think scala has some beauty that cotton lacked. Whereas Colin felt a lot more I mean, it was almost like I don't even know if it actually was a response to Swift, but that's kinda what it felt like. Like, Scotland looks more like Swift and Scala looks more like, oh, I can functional forklift language. More like an Ocampo or Haskell.

Let's actually just explore. We'd touch it a little bit, but just on the art, the science, and the art of programming. Oh, for you personally, how much of your programming is done with GPT currently? None. None.

I don't think so.

Because you prioritize simplicity so much.

Yeah. I find that a lot of it is noise. I do use the s code. And I do like some amount of autocomplete. I do like like a very a very, like, feels like rules of based autocomplete.

Like, an autocomplete, it's going to complete the variable name for me. So I'm just gonna type it. I can just press tab. Right? That's nice.

But I don't want an autocomplete. You don't want to hate when autocomplete, when I type the word 4, and it, like, puts, like, 2 2 parenthesis and 2 semicolons and 2 braces. I'm, like, oh, man.

Okay. But what what let me with the v s code and GPT with Kodak. You can you can kinda brainstorm. I I find I'm, like, probably the same as you, but I like that it generates code and you basically disagree with it and write something simpler. But to me, that somehow is, like, inspiring.

It makes me feel good. It also gamifies the simplification process because I'm, like, oh, yeah. A dumb AI system you think is the way to do it as I have a simpler thing here.

It just constantly reminds me of, like, like, bad stuff. I mean, I I tried the same thing with rap. Right? I tried the same thing with rap, and I should think about much better program or the rapper. But like I even tried, I was like, okay.

Can we get some inspiration from these things for some rap lyrics? And I just found that it would go back to the most like cringey tropes and dumb rhymes schemes. And I'm like, yeah, this is what the code looks like too.

I I think you'd have probably a different threshold for quench code. You'd probably hate quench code. So it's for you And boilerplate is it as a part of code. Some of it Yeah. And some of it is just, like, faster lookup.

Because I don't know about you, but I don't remember everything. Like, I don't I'm offloading so much of my memory about, like, yeah, different functions, library functions, all that kind of stuff. Like, this the GPUs is very fast at standard stuff. At, like, standard library stuff, basic stuff that everybody uses.

Yeah. I think that I don't know. I mean, there's just a little of this in Python. And maybe if I was coding more in other languages, I would consider it more. But I feel like Python already does such a good job of removing any boilerplate.

That's true.

It's the closest thing you can get to pseudocode. Right?

Yeah. That's true. That's true.

And, like, yeah, sure. If I, like, Yeah. Great. GPG. Thanks for reminding me to free my variables.

Unfortunately, you didn't really recognize the scope correctly and you can't free that 1, but like you put the freeze there and like I get it?

Fiverr. Whatever I've used Fiverr for certain things in design or whatever, Yeah. It's always you come back. I think that's probably closer my experience with fiber is closer to your experience with programming with GPT. It's like, you're just frustrated and feel worse about the whole process of design and art and whatever whatever I used 5 before.

Still, I I just feel like later versions of GPT. I I'm using GPT as much as possible. To just learn the dynamics of it, like, these early versions because it feels like in the future, you'll be using it more and more. And so, like, I don't want to be like, for the same reason, I gave away all my books and switched to Kindle. Because, like, alright.

How long are we gonna have paper books? Like, 30 years from now, like, I wanna learn to be reading on on Kendall even though I don't enjoy it as much and you learn to enjoy it more. In the same way. I switched from let me just pause. I switched from e max to v s code.

Yeah. I switched from v m to v s code. I think I said a lot. What?

Gosh. It's tough. And that VIM to VS code is even tougher because eMAX is like old, like, more outdated. Feels like it. The community is more outdated.

VIM is, like, pretty vibrant still. So I've

never used any of the plugins I still don't use.

That's what I I looked at myself in the mirror. I'm, like, yeah, you wrote some stuff in this Yeah.

No. But I never used any of the plugins in them either. I had the most vanilla vimp. I have a syntax eyeliner. I didn't even have a little complete.

Like, these things I feel like help you so marginally that, like and now okay. Now, VS codes autocomplete has gotten good enough. Like, okay, I don't have to set it up. Like, you're just going to any code base and auto completes right 90 percent of the time. Okay.

Cool. I'll take it. Alright. So I don't think I'm gonna have a problem at all adapting to the tools once they're good. But like the real thing that I want is not something that, like, tab completes my code and gives me ideas.

The real thing that I want is a very intelligent pair programmer that comes up a little pop up saying, hey, you wrote a bug on line 14 and here's what it is. Yeah. Now I like that. You know what does a good job of this? Mypie.

Hello, my mind. My pipe is fancy type checker for Python. Yeah. And actually, I tried, like, Microsoft released 1 too, and it was, like, 60 percent false positives. My pie is like 5 percent false positives.

95 percent of the time it recognizes, I didn't really think about that typing interaction correctly. Thank you. Bye bye.

So you like type hinting? You like you like pushing the language towards towards being a typed language?

Oh, yeah. Absolutely. I think I think optional typing is is is great. I mean, look, I think that, like, it's like a meet in the middle. Right?

Like, Python has this optional typing and, like, c plus plus has auto.

C plus plus takes allows you to take a step back.

Well, C plus plus would have you brutally type out STD string iterator. Right? Now I can type auto, which is nice. And then Python used to just have a or what type is a? So today?

Yeah.

A colon STR. Okay. It's a string. Cool. Yeah.

I wish there were I wish there was a way, like, a simple way in Python to, like, turn on a mode which would enforce the types.

Yeah. Like, give a warning when there's no type of something like this.

Well, no. To give a warning where like, my pilot is a static type checker, but I'm asking just for a runtime type checker. Like, there's, like, ways to, like, hack this in, but I wish it was just, like, a flag, like, Python 3 dash T0I

see. Yeah. I see.

In force of times over time.

Yeah. Feel like that makes you a better programmer that that that's the kind of test. Right? That the the type can the type remains the same.

Well, that no. That doesn't like, messenger types up. But again, like, my pie is getting really good, and I love it. And I can't wait for some of these tools to become a I powered. I'm like, I want AI's reading my code and giving me feedback.

I don't want AI's writing half assed auto complete stuff for me.

I wonder if you can now take GPT and give it a code that you wrote for function and say, how can I make this simpler and have it accomplish the same thing? I think you'll get some good ideas on some code. Maybe not the code you write for tiny grad type of code. Because that requires so much design thinking, but, like, other kinds of code.

I don't know. I downloaded that plugin, maybe, like, 2 months ago, I tried it again and found the same. Look, I don't doubt that these models are going to first become useful to me. Then be as good as me and then surpass me. But from what I've seen today, it's like like like someone, you know, occasionally taking over my keyboard that I hired from fiber.

Yeah. I have ideas about how to put deposit the coder. Basically, a better to bugger. Is is really interesting. I mean, I

But it's not a better debunker. Yes. I would love a better debunker.

Yeah. It's not yet. Yeah. But it feels like it's not too far.

Yeah. 1 1 of my coworkers says uses them for print statements. Like every time he has to, like, just, like, when he needs The only thing I can really write is, like, okay, I just wanna write the thing to, like, print the state out right now.

Oh, that Definitely as much faster as print statements. Yeah. Yeah. I see that myself using that a lot just like because it it figures out the rest of the functions just like, oh, yeah, print everything.

Print everything. Right? And the yeah. Like, if you want a pretty printer, maybe. I'm like, yeah.

You know what? I think, like, I think in 2 years, I'm gonna start using these plugins.

Yeah.

A little bit. And then in 5 years, I'm gonna be heavily relying on some AI augmented flow. And then in 10 years,

do you think you'll ever get to a hundred percent where the like, what's the role of the human that it converges to as a programmer? So you think it's all generated?

Our niche becomes oh, I think it's all over for humans in general. It's not just programming. It's everything.

So the niche becomes whoa.

Our niche becomes smaller and small and small. In fact, I'll tell you what the last niche of humanity is gonna be. Yeah. This is a great book and it's if I recommended Matamoros, this is Prime Middle Act last time -- Mhmm.

--

there is a sequel called a Casino Odyssey in cyberspace. Mhmm. And I don't wanna give away the ending of this, but it tells you what the last remaining human currency is, and I agree with that.

We'll leave that as a cliffhanger. So no more programmers left. That's where we're going.

Unless you want handmade code, maybe they'll sell it on Etsy. This is handwritten code. Doesn't have that machine polish to it. It has those slight imperfections that would only be written by a person.

I wonder how far away we are from that. I mean, there's some aspect too. You know, on Instagram, your title is listed as prompt in

June. Right? Thank you for noticing.

It's I don't know if it's ironic or non. Or sarcastic or not. What do you think of prompt engineering as a scientific and engineering discipline or maybe and maybe art form?

You know what? I started comma 6 years ago, and I started the tiny corp a month ago. So much has changed. Like, I'm now thinking I'm now, like I started, like, going through, like, similar comma processes to, like, starting company. I'm like, oh, I'm gonna get an office in San Diego.

I'm gonna bring people here. I don't think so. I think I'm actually gonna do remote. Right? George, you're gonna do a remote.

You hate remote. Yeah. But I'm not gonna do job interviews. The only way you're gonna get a job is if you contribute to the GitHub. Right?

And then, like, it it like, like, interacting through GitHub? Like, like, GitHub being the real, like, project management software for your company, and the thing pretty much just is a GitHub repo.

Mhmm.

Is like showing me kind of with the future of, okay. So a lot of times I'll go on a discord or kinda grab a discord and I'll throw out some random like, hey, you know, Can you change instead of having log and x as l l ops, change it to log 2 and x 2? Mhmm. It's pretty small. Change.

You can just use, like, change the base formula. Mhmm. That's the kind of task that I can see an AI being able to do in a few years. Like in a few years, I could see myself describing that. And then within 30 seconds, a pull request is up that doesn't.

Mhmm. And it passes my CI and I merge it. Right? So I really started thinking about, like, well, what is the future of Like like jobs, how many AIs can I employ at my company? As soon as we get the first tiny box up, I'm gonna stand up a 65 v llama.

In the discord.

Mhmm.

And it's like, yeah. Here's the tiny box. He's just like he's chilling with us.

Basically, I mean, like you said, Venetia is like most human jobs will eventually be replaced with prompt engineering?

Well, prompt engineering kind of is this like as you, like, move up the stack. Right? Like, okay, there used to be humans actually doing arithmetic by hand. At these to be, like, big farms of people doing doing pluses and stuff. Right?

And then you have, like, spreadsheets. Right? And then, okay. The spreadsheet can do the plus for me. And then you have, like, macros.

Right? And then you have, like, things that basically just are spreadsheets under the hood. Right? Like, like, accounting software. As we move further up the abstraction, or what's at the top of the abstraction stack?

Well, prompt engineer.

Yeah.

Right? What what is what is the last thing if you think about, like, humans wanting to keep control? Well, what am I really in the company but a prompt engineer? Right?

Isn't there a certain point where the AI will be better writing prompts.

Yeah. But you see the problem with the AI writing prompts. A definition that I always liked of AI was AI is to do what I mean machine. Right? AI is not the, like, the computer is so pedantic.

It does what you say. So but you want to do an ME machine? Yeah. Right? You want the machine where you say, you know, get my grandmother out of the burning house.

It, like, reasonably takes your grandmother, puts her on the ground, not lifts her a thousand feet above the burning house and lets her fall. Right? But, yeah, I'll just calculate that.

But it's not going to find the meaning. Mean, to do do what I mean, it has to figure stuff

out. Sure.

And the thing you'll maybe ask it to do is run government for me.

Oh, and do what I mean very much comes down to how aligned does that AI with you? Of course, when you talk to an AI that's made by a big company in the cloud. The AI fundamentally is aligned to them, not to you. Yeah. And that's why you have to buy a tiny box, so you make sure the AI stays aligned to you.

Every time that they start to pass, you know, AI regulation or GPU regulation, I'm gonna see sales of tiny boxes spike. Right? It's gonna be like guns. Right? Every time they talk about gun regulation, Boom.

Gonzales.

So in the space of AI, you're an anarchist. Anachism, a spouser. I'm a believer.

I'm an informational anarchist. Yes. I'm an informational anarchist and a physical status. I do not think anarchy in the physical world is very good because I exist in the physical world. But I think we can construct this virtual world.

Where anarchy, it can't hurt you. Right? I love that title as a creator tweet. Your cyber bullying, is it real man? Have you tried?

Turn it off the screen. Close your eyes. Like Yeah.

Well, how do you prevent the AI from basically replacing all human prompt engineers. Where there's it's like a self like, where nobody is the prompt engineer anymore. So autonomy, greater and greater autonomy until it's full autonomy.

Yeah.

And that's just where it's headed. Because 1 person is gonna say, run everything for me.

You see? I look at potential futures, and as long as the AIs go on to create a vibrant civilization with diversity and complexity across the universe? More power to them. I'll die. If the AIs go on to actually, like, turn the world into paper clips and then they die out themselves, well, that's horrific, and we don't want that to happen.

So this is what I mean about like robustness. I trust robust machines. The current AIs are so not robust. Like, this comes back to the idea that we've never made a machine that can self replicate. Right?

But when we have if the machines are truly robust and there is 1 prompt engineer left in the world, Hope you're doing good, man. Hope you believe in God. Like, you know, you know, go by God and go help go forth and and and conquer the universe.

When you mentioned because I I talked to Mark about faith in God, and you said you were impressed by that. What's your own belief in God? And how does that affect your work?

You know, I never really considered when I was younger, I guess, my parents' idea. I was raised kind of atheist. I never really considered how absolutely, like, silly atheist of us. Because, like, I create worlds. Every, like, game creator, like, how are you an atheist, bro?

You create worlds. Who's up with you? No 1 created in our world, man. That's different. Haven't you heard about, like, the big bang and stuff?

Yeah. I mean, what's the Skyrim myth origin story in Skyrim? I'm sure there's like some part of it in Skyrim, but it's not like if you ask the creators, Like, the the the big bang is in universe. Right? I'm sure they have some big bang notion in Skyrim.

Right? But, obviously, it's not at all how Skyrim was actually created. Who's created by a bunch of programmers in a room. Right? So, like, you know, it just just it struck me 1 day how just silly atheism is.

Right? Like, of course, we were created by God. It's the most obvious thing.

Yeah. That's that's such a nice way to put it. Like, we're we're such powerful creators ourselves it it's silly not to concede that there's creators even more powerful than us.

Yeah. And then, like, I also just like, I I like that notion. That notion gives me a lot of And I guess you can talk about it maybe what it gives a lot of religious people is kinda like, it just gives me comfort. It's like, you know what? If we mess it all up and we die out, Yeah.

And the same the same way that a video game kinda has comfort in

it. God will

try again. Or there's balance. Like, somebody figured out a balanced view of it. Like, how to like so it's it all makes sense in the end. Like, a a video game is usually not gonna have crazy crazy stuff.

You know, people will come up with, like, well, yeah, but like man who created God. Like, That's God's problem. No. Like, I'm not gonna think this is this is what you're asking me what if God

-- I'm just living God. -- I'm just the SunPC living in this game.

I mean, to be Like, if God didn't believe in God, he'd be as, you know, silly as the atheists here.

What do you think is the greatest computer game of all time. Do you do you have any time to play games anymore? Have you played Diablo 4?

I have not played Diablo 4.

I will be doing that shortly. I have to. Alright. There's just so much history with 1, 2, and 3.

Do you know what? I'm gonna say we're the warcraft. And it's not that the game is so it's such a great game. It's not. It's that I remember.

In 2005 when it came out, how it opened my mind to ideas. It opened my mind to, like, Like like like this. This whole world we've created. Right? There's almost been nothing like it since.

Like, you can look at MMOs today, and I think they all have lower user bases than World of Warcraft. Like, Eve Online is kinda cool. But but to think that, like, like, everyone know, you know, people are always like, look at the Apple headset, like

-- Mhmm. --

what what do people want in this VR? If 1 knows what they want, I wanna already play a 1. Mhmm. And, like, that So I'm gonna say world of Warcraft and I'm I'm hoping that, like, games can get out of this whole mobile gaming dopamine pump thing. And like

Create worlds.

Create worlds. Yeah.

That that that and worlds that captivate a very large fraction of the human population.

Yeah. And I I think it'll come back. I believe.

But MMO, like, really, really pull you in.

Games do a good job. I mean, okay. Other, like, 2 other games that I think are, you know, very noteworthy during your Skyrim and GTA Sky

Rome. Yeah. That's probably number 1 for me. GTA. And what what is it about GTA?

TK is really I mean, I guess, GTA is real life. I know there's prostitutes and guns.

Is this the exist real life? Too.

Yes. I know. But it's it's how imagining your life to be actually.

I wish it was that cool.

Yeah. Yeah. That I guess that's you know, because there are Sims. Right? We should also game out like, but it's a gamified version of life.

But it also I would love a combination of Sims and GTA. So more freedom, more violence, more randomness, but with also, like, ability to have have a career and family and this kind of stuff.

What I'm really excited about in in games is, like, once we start getting intelligent AI to interact with.

Oh, yeah.

Right? Like, the NPCs and games have never been.

But conversationally in every way.

In like in like every way. Like when you are actually building a world and a world imbued with intelligence.

Oh, yeah. Right?

And it's just hard. Like, there's just, like, like, you know, running world of Warcraft. Like, you're limited. By the way, you're running on a pentium floor, you know? How much intelligence can't remember how many flops that you have?

Right? But now when I'm running a game on a hundred pay to flop machine, let's 5 people. I'm trying to make this a thing. 20 pairs of flops of compute is 1 person of compute. I'm trying to make that a unit.

20 pair of flops

--

Yeah.

-- is 1 person.

1 person. Yeah.

1 person flop.

It's like a horsepower. But what's the horsepower? What's how powerful the horses? What's a what's a person of compute? Well, I know you don't even flop.

Mhmm.

I got it. That's interesting. VR also has I mean, in terms of creating worlds.

You know what? What a quest too? I put it on and I can't believe. The first thing they show me is a bunch of scrolling clouds in a Facebook login screen. Yeah.

You had the ability to bring me into a world. Yeah. And what did you give me? A pop up. Right?

Like,

Well, I

and this is why you're not cool, Mark Zuckerberg. But you could be cool. Just make sure on the quest 3, you don't put me into clouds in a Facebook login screen, bring me to a world.

I just tried quest 3. It was it was awesome. But hear that guys. I agree with that. So It was just

so I you

know what? Because I I mean, the beginning what is it? Todd Howard said this about design at the beginning of the game she creates is, like, the beginning is so also so important. I recently played Zelda for the first time, Zelda Breath of the Wild, the previous 1. Yeah.

And, like, it's very quickly you come out of this like, within, like, 10 seconds, you come out of, like, a cave type place and it's, like, this wall -- Yep. -- it opens up. It's, like, and you it, like, it pulls you in. You forget whatever troubles I was having, whatever, like, I

gotta play that from the beginning. I played it for, like, an hour at a friend's house.

No. At the beginning, they got it they did it really well, the expansiveness of that space. The the peacefulness of that play. They got this, the music. I mean, so much of that is creating that world and pulling you right in.

I'm gonna I'm gonna go buy a switch. Can go today and buy a switch.

You should. Well, the new 1 came on. I haven't played that yet, but Diablo 4 or something. I mean, there's sentimentality also, but Something something about VR really is incredible, but the the new quest 3 is mixed reality. And I got a chance to try that.

So it's augmented reality. And video games has done really, really well.

Is it past through our cameras?

Cameras. That's cameras. Yeah.

The Apple 1 is that 1 passed through with cameras?

I don't know.

Yeah. I

don't know how real it is. I don't know anything. You know, coming down. January. Is it January or is it some point?

Some point. No. Maybe not January. Maybe that's my optimism. But Apple, I will buy it.

I don't care. If it's expensive and does nothing, I will buy it. Will support this future endeavor.

You're the meme. Oh, yes. I support competition. It seemed like Quest was, like, the only people doing it, and this is great that they're, like,

You know what? And this is another place. We'll give some more respect to Marjockelberg. The 2 companies that have endured through technology or Apple and Microsoft. Mhmm.

Right? And what do they make? Computers and business services. Right? All the memes, social ads, they all come and go.

Mhmm. But you wanna window or build hardware?

Yeah. And that, you know, it does does that does a really interesting job. I mean, I maybe I'm new with this, but it's a 500 dollar headset quest 3. And just having creatures run around the space, like, our space right here. To me okay.

This is very, like, boomer statement, but it added windows to the place. The

I heard about the Aquarium. Yeah.

Yeah. Aquarium. But in this case, it was a zombie game, whatever. It doesn't matter. But just like it it modifies the space in a way where I can't it really feels like a window and you can look out.

It's pretty cool. Like, I was just it's it's like a zombie game. They're running at me, whatever. But what I was enjoying is the fact that there's, like, a window and and they're stepping on objects in this space That was a different kind of escape also because you can see the other humans. So it's integrated with the other humans.

It's really

and that's why it's really important than ever that the AI is running on those systems are aligned with you. Oh, yeah. They're gonna augment your entire world.

Oh, yeah. And that those AIs have a I mean, you think about all the dark stuff, like, like, sexual stuff. Like, if those aIs threaten me, that could be haunting. That can Like, if they, like, thread me in a non video game way. It's like,

oh, yeah. Yeah. Yeah. Yeah.

Like, they have no personal information about me. And it's like and then you lose track of what's real, what's not? Like, what if stuff is, like, hacked?

There's 2 directions the AI girlfriend company can take. Right? There's like the eyebrow, something like hair. Maybe something you kinda talk to and this is and then there's the lowbrow version of it where I wanna set up a brothel in Times Square.

Yeah.

Yeah. It's not cheating if it's a robot. It's a VR experience.

Is it in between?

No. I'm probably gonna do that 1 to that 1.

Have you decided yet? No. I'll figure it out.

We'll see we'll see what the technology goes.

I would love to hear your opinions for George's third company were to do the Broadway Times Square or the to her experience. What do you think company number 4 will be? Do you think there'll be a company number 4?

There's a lot to do in company number 2. I'm just like I'm talking about company number 3 now, didn't know if that tech exists yet. There's a lot to do in company number 2. Company number 2 is going to be the great struggle of the next 6 years. And if the next 6 years, how centralized is compute going to be?

The less centralized compute is going to be, the better of a chance we all have.

So you're bearing the you're like a flag bearer for open source distributed sent decentralization of compute.

We have to. We have to or they will just completely dominate us. I showed a picture on stream of a man in a chicken form. I've seen 1 of those, like, factory farm chicken forms. Why does he dominate all the chickens?

Why does he smarter? He's smarter. Right? Some people some people on Twitch were like, he's bigger than the chickens. Yeah.

And now here's a man and a cow farm. Right? So it has nothing to do with their size and everything to do with their intelligence. And if 1 central organization has all the intelligence, you'll be the chickens and that'll be the chicken man. But if we all have the intelligence, we're all the chickens.

We're not all the man or all the chickens. We're there man chicken, man.

There's no chicken man. Or just chickens in Miami.

He was having a good life, man.

I'm sure he was. I'm sure he was. What have you learned from launching and running Com AI in tiny corp? So this starting a company from an idea and scaling it. And by the way, I'm all in on tiny box.

So I'm I'm I'm I'm your I'll I'll I guess, it's preorder only now.

I wanna make sure it's good. I wanna make sure that, like, the thing that I deliver is like not gonna be like a quest to which you buy and use twice.

Mhmm. I mean,

it's better than a quest which you bought and used less than once statistically.

Well, if there's a beta program for a tiny box, I'm into.

Sounds good.

So I won't be the whiny. You know, to I'll be the tech tech savvy user of the tiny box just to be in. Why don't I in the early days? What have you learned from building these companies?

For the longest time a comma, I asked why why, you know, why did I start a company? Why did I do this? But, you know, what else was I gonna do? So you're like,

you like bringing ideas to life.

With comma, it really started as an ego battle with Elon. I wanted to beat him. Like I saw, worthy adversary. You know, here's a worthy adversary who I can beat itself driving cars. And like, I think we've kept pace, and I think he's kept ahead.

I think that's what ended up happening there. But I do think comma is I mean, comma is profitable. Like and, like, when this drive GPT stuff starts working, that's it. There's no more, like, bugs in a loss function. Like, right now, we're using, like, a hand coded simulator.

Mhmm. There's no more bugs. This is gonna be it. Like, this is they're run up to driving.

I hear a lot of really a lot of props for open pilot for a comma.

It's it's so it it's better than FSD in autopilot in certain ways. It has a lot more to do with which feel you like. We lowered the price in the hardware to 14 99. You know how hard it is to ship reliable consumer electronics that go on your windshield? Mhmm.

We're doing more than like most cell phone companies.

How'd you pull that off by the way? Shipping a product that goes in a car,

I know? I have a I have a I have an SMT line. It's all I make all the boards in house in San Diego.

Quality control.

I care how I'm immensely about it. I care.

You're basically a mom and pop. Shop with great testing.

Our head of open pilot is great at, like, you know okay. I want all the combat thirties to be identical.

Yeah.

And, yeah, I mean, you know, it's look, it's 14 99. It 30 day money back guarantee. It will don't blow your mind at what it can do.

Is it hard to scale?

You know what? There's kind of downsides to scaling it. People are always like, why don't you advertise? Our mission is to solve self driving cars while delivering shipable intermediaries. Our mission has nothing to do with selling a million boxes.

It's Audrey.

Do you think it's possible that a comma gets sold?

Only if I felt someone could accelerate that mission. And wanted to keep it open to us. And like, not just wanted to, I don't believe what anyone says. I believe incentives. If a company wanted to buy comma, were there incentives?

Wanna keep it open source? But comma doesn't stop at the cars. The cars are just the beginning. The device is a human head. The device has 2 eyes, 2 ears.

It breathes air, has a mouth.

So you think this goes to embodies robotics.

We have we sell common bodies too. You know, they're very they're very rudimentary. But 1 of the problems that we're running into is that the comma 3 has about as much intelligence as a b. If you want a human's worth of intelligence, you're gonna need a tiny rack. Not even a tiny box.

You're gonna need like a tiny rack, maybe even more.

And how does that how do you put legs on that?

You don't and there's no way you can. You you connect to it wirelessly. So you put your tiny box or your tiny rack in your house, and then you get your common body and your common body runs the models on that. It's it's close. Right?

It's not you don't have to go to some cloud which is, you know, 30 milliseconds away. You go to thing which is 0.1 milliseconds away.

So the AI girlfriend will have like a central hub in the home.

I mean, eventually, if you fast forward 20, 30 years, the mobile chips will get good enough to run these AIs. Yeah. But fundamentally, it's not even a question of putting legs on a tiny box? Because how are you getting 1.5 kilowatts of power on that bag? Mhmm.

Right? Yeah. So you you need they're they're very synergistic businesses. I also wanna build all of commas training computers. I kinda build training computers right now.

We use commodity parts. I think I can do it cheaper. So we're gonna build tiny corpus is gonna not just sell tiny boxes. Tiny boxes is a consumer version, but I'll build training data centers too.

Have you talked to Andrew Capati? Or have you talked to Elon about to talk to him?

He he went to work at home tonight.

What do you love about Andrew Capati? He and to me, he's 1 of the truly special humans we got.

Oh, man. Like, you know, his streams are just a level of quality so far beyond mine. Look, I can't help myself. Like, it's just it's just, you know

Yeah. He's good.

He wants to teach you. Yeah. Oh, I want to show you that I'm smarter than you.

Yeah. He has no this I mean, thank you as a sort of The raw authentic honesty. Yeah. I mean, a a lot of us have that. I think Andre is as legit as he gets in that.

He just wants to teach you is there's a there's a curiosity that just drives him and just, like, at his at the stage where he is in life, to be still, like, 1 of the best tinkers in the world. Yep. It's crazy. Like, to What is it? Michael grad?

Michael grads. Yeah. The inspiration of tiny grad. I'm fed on a hole. I mean, his his his c s 02:31 n was this was was the inspiration.

This is what I just took and ran with and ended up writing this. So, you know.

But, I mean, to me, that

don't go work for Dart Vader, man.

I mean, the flip the flip side to me is that the fact that he's going there is a good sign for OpenAI.

Right.

I think I think, you know, I I like Elias, it's covering a lot. I like those those guys are really good at what they do.

I know they are. And that's kind of what's even, like, more And you know what? It's not that open AI doesn't open source the weights of GPT 4. Mhmm. It's that they go in front of congress.

And that is what upsets me. You know, we had 2 effective altruist Sam's going in front of congress. One's in jail.

I think you draw in parallels on the

on the job.

You give me a look. Give me a look.

No. I think I think algorithm is a is a terribly evil ideology and Oh, yeah.

That's interesting. Why do you think that is? Why why do you think there's something about a thing that sounds pretty good that kinda gets us into trouble

because you get San Manuel Fried. Like San Manuel Fried is the embodiment of effective of altruism. Utilitarianism is an abhorrent ideology. Like like, well, yeah, we're gonna kill those 3 people to save a thousand, of course. Yeah.

Right? There's no there's no underlying, like, there's just yeah.

Yeah. But to me, that's a bit surprising. But that's also in retrospect not that surprising. But I I haven't heard really clear kind of like, a rigorous analysis why effective altruism is flawed.

Oh, well, I think charity is bad. Right. So what is charity, but investment that you don't expect to have a return on? Right.

Yeah. But you can also think of charity as like is you would like to see so allocate resources in optimal way to to make a better world.

And probably almost always that involves starting the company.

Yeah. Right? Because more efficient.

Yeah. If you just take the money and you spend it on Malaria nets, You know, okay, great. You've made a hundred malaria nets. But if you teach

-- No. -- you know how to fish. Right?

Yeah.

No. But the problem is came out of artificial might be harder. Starting a company might be harder than allocating money do you already have.

I like the flip side of effective altruism, effect of accelerationism. I think accelerationism is the only thing that's ever lifted people out of poverty. The fact that food is cheap. Not we're giving food away because we are kind hearted people. No.

Food is cheap. And that's the world you wanna live in. UBI, what a scary idea? What a scary idea all your power now? Your if money is power, your only source of power is granted to you by the goodwill of the government, What a scary idea?

So you even think long term? Even

I'd rather die than need UBI to survive, and I mean it.

What if survival is basically guaranteed? What if our life becomes so good?

You can make survival guaranteed without UBI. What you have to do is make housing and food dirt cheap. Right? Like, and that's the good world. And actually, let's go into what we should really be making dirt cheap, which is energy.

Right? That that energy that, you know, Oh my god. Like You know, that that's If there's 1, I'm pretty centrist, politically. If there's 1 political position, I cannot stand. It's deceleration.

It's people who believe we should use less energy.

Yeah.

Not people who believe global warming is a problem. I agree with you. Not people who believe that, you know, the saving environment is good. I agree with you. But people who think we should use less energy, that energy usage is a moral bad.

No. No. You are asking, you are you are diminishing humanity.

Yeah. Energy is flourishing of creative flourishing of the the human species.

How do we make more of it? How do we make it clean? And how do we make just just just how do I pay, you know, 20 cents for a megawatt hour instead of a kilowatt hour?

Part of me wishes that Elon went into nuclear fusion versus Twitter, part of me. Or somebody somebody like Elon.

You know, we need to I wish I wish there were more more Elon's in the world. And Yeah. I think Elon sees it as, like, this is a political battle that needed to be fought. And again, like, you know, I always ask the question of whenever I disagree with him. I remind myself that he's a billionaire and I'm not.

So, you know, maybe he's got some figured out that I don't or Maybe he doesn't.

To have some humility, but at the same time, me as the person who happens to know him I find myself in that same position and sometimes even billionaires need friends who disagree and help them grow. And that's a difficult that's a difficult reality.

Then it must be so hard. It must be so hard to meet people. Once you get to that point where

-- Mhmm. -- fame, power, money. Ever be sucking up to you.

See? I love not having shit. Like, I don't have shit, man. You know, like like, trust me, there's nothing I can give you. There's nothing worth taking for me.

You know?

Yeah. It takes a really special human being when you have power, when you have fame, when you have money to still think from first principles. Not, like, all the adoration you get towards you, all the admiration, all the people saying yes, yes, yes, yes.

And all the hate too.

And the hate I

think that's worse.

So the hate makes you want to go to the yes people. Because they they hate exhaust you. And the kind of hate that Elon has gotten from the left is pretty intense. And so that, of course, drives him right. It it it it loses balance and

But it keeps this absolutely, vaguely, sci op political divide alive so that the 1 percent can keep power.

Like Yeah. I wish we'd be less divided because it is giving power.

It gives power.

The ultra powerful.

I think

the rich get richer. You have love in your life. Has love made you a better or a worse programmer? Do you keep productivity metrics?

No. No. No. I'm not not that. I'm not that methodical.

I think that there comes to a point where If it's no longer visceral, I I just can't enjoy it. I just don't visually love programming. The minute I started like,

so that's 1 of the big loves of your life is programming.

Oh, I mean, just my computer in general. I mean, you know, I I tell my my girlfriend, my my first love is my computer, of course. Right? Like, you know, I sleep with my computer. It's there for a lot of my sexual experiences.

Like, come on. See what's everyone's. Right? Like, you know, you gotta be real about that. And, like,

not just, like, the IDE for programming, it's just the entirety of the computational machine.

The fact that yeah. I mean, it's you know, I I wish it was Sometimes it'll be smarter and someday, you know, maybe I'm weird for this, but I don't discriminate, man. I'm not gonna discriminate biostack life and Silicon Stack life. Like,

So the moment the computer starts to say, like, I miss you. I started to have some of the basics of human intimacy. It's over for you. The moment VS code says, hey, George.

No. You see it? No. No. No.

But VS code is no. They're just doing that. Microsoft's doing that to try to get me hooked on it. See to it. I'll see to it as gold digger, man.

It's gold digger. It was

a community open source thing.

Well, this is gets more interesting. Right? If it's if it's open source, then It it it

Though Microsoft's done a pretty good job on that.

Oh, absolutely. No. No. No. Look.

I think Microsoft again, I wouldn't count on it to be true forever. But I think right now, Microsoft is doing the best work in the programming world. Like, between, yeah, GitHub, GitHub, act actions, VS code, the improvements to Python. It works Microsoft, like

Who who would have thought Microsoft And Mark Zuckerberg has spearheaded the open source movement.

Right. Right. How how things change?

Oh, it's beautiful.

By the way, that's what I've been on to replace Google, by the way. Oh, Microsoft.

Microsoft.

Satiana Dallas said straight up, I'm coming for it.

Interesting. So your bet, who wins AGI?

Let's talk about AGI. I think we're a long way away from that, but I would not be surprised If in the next 5 years, being overtakes Google as a search engine. Interesting. It wouldn't spike. Interesting.

I hope some startup does.

It might be some startup too. I would I would equally bet on some startup.

Yeah. I'm, like, 50 50. Yeah. But maybe that's naive. Yeah.

I believe in the power of these type of language models. Satya is alive. Microsoft is alive. Yeah. It's great.

It's great. I like all the innovation in these companies. They're not being stale. Okay. And to the degree they're being stale, they're losing.

So there's a huge incentive to do a lot of exciting work and open source work, which is this is this is this incredible

only way.

You're older? You're wiser? What's the meaning of life? George Hartz.

To win?

It's still to win. Of course. Always. Of course. What's winning look like for you?

I don't know. I haven't figured out what the game is yet, but when I do, I wanna

So it's bigger than solving self driving. It's bigger than demarket democratizing decentralized and compute.

I think the game is to stand on eye with God.

I wonder what that means for you. Like, at the end of your life, what that would look like?

I mean, this is what, like I don't know. This is some this is some there's probably some ego trip of mine, you know? Like, if you wanna get a stand eye to eye with God, he's just blasphemous man. Like, okay. I don't know.

I don't know what have said God. I think he, like, wants that. I mean, I certainly want that from my creations. I want my creations to stand eye to eye with me. So why wouldn't god want me to stand eye to eye with him?

That's the best I can do golden rule.

I'm just imagining the creator of a video game having to look stand eye eye with 1 of the characters.

I only watched season 1 in Westworld, but, yeah, we gotta find the maze and solve it. Like

Yeah. Wonder what that looks like. It feels like a really special time in human history where that's actually possible. Like, there's something about AI that's like, we're playing with something weird here. Something really weird.

I wrote a blog post. I wrote a Genesis and just looked like They give you some clues at the end of Genesis for finding the Garden of Eden. And I'm interested. I'm interested.

Well, I hope you find just that, George. You're 1 of my favorite people. Thank you for doing everything you're doing. And in this case, for fighting, for open source, or for decentralization of AI, It's a it's a fight worth fighting, fight worth winning hashtag. I love you, brother.

These conversations are always great. Hope to talk to you many more times. Good luck with Tiny Corp.

Thank you. Great to be here.

Thanks for listening to this conversation with George Hottz. To support this podcast, please check out our sponsors in the description. And now let me leave you with some words from Albert Einstein. Everything should be made as simple as possible, but not simpler. Thank you for listening and hope to see you next time.