The following is a conversation with Kyle Vogt.

He's the president and the CTO of Cruise Automation, leading an effort to solve 1 of the biggest robotics challenges of our time, vehicle automation.

He's a co-founder of 2 successful companies, Twitch and Cruise, that have each sold for a billion dollars.

And he's a great example of the innovative spirit that flourishes in Silicon Valley.

And now is facing an interesting and exciting challenge of matching that spirit with the mass production and the safety-centric culture of a major automaker, like General Motors.

This conversation is part of the MIT Artificial General Intelligence series and the Artificial Intelligence podcast.

If you enjoy it, please subscribe on YouTube, iTunes, or simply connect with me on Twitter, at Lex Friedman, spelled F-R-I-D.

And now, here's my conversation with Kyle Vogt.

Yeah, and I just saw that picture you had hidden over there so I'm a little bit worried about that now.

So in high school in Kansas City, you joined Shawnee Mission North High School Robotics Team.

That was the only high school in the area that had a teacher who was willing to sponsor a FIRST Robotics team.

I was trying to look super cool and intense.

So we're standing there with a welded steel frame and looking tough.

Well, I think I've been trying to figure this out for a while but I've always liked building things with Legos.

And when I was really, really young, I wanted the Legos that had motors and other things.

And then, you know, Lego Mindstorms came out and for the first time you could program Lego contraptions.

And I think things just sort of snowballed from that.

But I remember seeing the BattleBots TV show on Comedy Central and thinking that is the coolest thing in the world, I wanna be a part of that.

And not knowing a whole lot about how to build these 200 pound fighting robots.

So I sort of obsessively poured over the internet forums where all the creators for BattleBots would sort of hang out and talk about, you know, document their build progress and everything.

And I think I read, I must have read like, you know, tens of thousands of forum posts from basically everything that was out there on what these people were doing.

And eventually, like sort of triangulated how to put some of these things together and ended up doing BattleBots, which was, you know, I was like 13 or 14, which was pretty awesome.

I'm not sure if the show is still running, but so BattleBots is, there's not an artificial intelligence component.

And it's almost like a mechanical engineering challenge of building things that can be broken.

So, and I think that they allowed some limited form of autonomy, but in a 2 minute match, and the way these things ran, you're really doing yourself a disservice by trying to automate it versus just, you know, do the practical thing, which is drive it yourself.

And there's an entertainment aspect, just going on YouTube, there's like, some of them wield an ax, some

Was it the dream to create like Frankenstein and sentient being or was it just like the Lego?

I mean that that was just building something I think the the idea of you know This is this radio-controlled machine that that can do various things if it has like a weapon or something was pretty interesting.

It's, it doesn't have the same appeal as, you know, autonomous robots, which I, which I, you know, sort of gravitated towards later on, but it was definitely an engineering challenge because everything you did in that competition was pushing components to their limits.

So we would buy like these $40 DC motors that came out of a winch, like on the front of a pickup truck or something.

And we'd power the car with those and we'd run them at like double or triple their rated voltage.

So they immediately start overheating, but for that 2 minute match, you can get a significant increase in the power output of those motors before they burn out.

And so you're doing the same thing for your battery packs, all the materials in the system.

And I think there was something intrinsically interesting about just seeing like where things break.

And did you offline see where they break?

did you know 2 minutes or was there a reckless, let's just go with it

We lost all of our matches in the first round.

The 1 I built first, both of them were these wedge-shaped robots, because a wedge, even though it's sort of boring to look at, is extremely effective.

You drive towards another robot and the front edge of it gets under them and then they sort of flip over, kind of like a door stopper.

And the first 1 had a pneumatic polished stainless steel spike on the front that would shoot out about 8 inches.

Pretty ineffective actually, but it looks cool.

No, it was just to try to poke holes in the other robot and then the second time I did it, which is the following, I think maybe 18 months later, we had a titanium axe with a hardened steel tip on it that was powered by a hydraulic cylinder which we were activating with liquid CO2, which had its own set of problems.

So great, so that's kind of on the hardware side.

I mean, at a certain point, there must have been born a fascination on the software side.

So what was the first piece of code you've written?

Yeah, well, I remember, I think maybe when I was in third or fourth grade, the school I was at, elementary school, had a bunch of Apple II computers and we'd play games on those.

And I remember every once in a while something would would would crash or wouldn't start up correctly And it would dump you out to what I later learned was like sort of a command prompt.

And my teacher would come over and type, I actually remember this to this day for some reason, like PR number 6 or PR pound 6, which is peripheral 6, which is the disk drive, which would fire up the disk and load the program.

And I just remember thinking, wow, she's like a hacker.

Like teach me these, these codes, error codes is what I called them at the time, but she had no interest in that.

And so it wasn't until I think about fifth grade that I had a, a school where you could actually go on these Apple twos and learn to program.

And so it was all in basic, you know, where every line, you know, the line numbers are all number that every line is numbered and you have to like leave enough space between the numbers so that if you want to tweak your code, you go back and if the first line was 10 and the second line is 20, now you have to go back and insert 15 and if you need to add code in front of that, you know, 11 or 12 and you hope you don't run out of line numbers and have to redo the whole thing.

Yeah, go-to and it's very basic, maybe hence the name, but a lot of fun and that was like, That was, you know, that's when, you know, when you first program, you see the magic of it.

It's like, it just, just like this world opens up with, you know, endless possibilities for the things you could build or accomplish with that computer.

So even starting with basic and then what, C++ throughout, what did you, was there a computer program and computer science classes in high school?

Not where I went, so I was self-taught, but I did a lot of programming.

The thing that sort of pushed me in the path of eventually working on self-driving cars was actually 1 of these really long trips driving from my house in Kansas to I think Las Vegas where we did the BattleBots competition and I had just gotten my I think my learner's permit or early driver's permit And so I was driving this 10 hour stretch across Western Kansas where it's just, you're going straight on a highway and it is mind numbingly boring.

And I remember thinking even then with my sort of mediocre programming background that this is something that a computer can do, right?

Let's take a picture of the road, let's find the yellow lane markers and steer the wheel.

And later I'd come to realize this had been done since the 80s or the 70s or even earlier, but I still wanted to do it.

And sort of immediately after that trip, switched from sort of BattleBots, which is more radio controlled machines, to thinking about building autonomous vehicles of some scale.

Start off with really small electric ones and then progress to what we're doing now.

So what was your view of artificial intelligence at that point?

So this is before, there's been waves in artificial intelligence, right?

The current wave with deep learning makes people believe that you can solve in a really rich, deep way the computer vision perception problem.

But like in, before the deep learning craze, you know, how do you think about, how would you even go about building a thing that perceives itself in the world, localize itself in the world, moves around the world?

Well, prior to deep neural networks or convolutional neural nets, these modern techniques we have, or at least ones that are in use today, it was all heuristic space.

And so like old school image processing, and I think extracting yellow lane markers out of an image of a road is 1 of the problems that lends itself reasonably well to those heuristic based methods, you know, like just do a threshold on the color yellow and then try to fit some lines to that using a puff transform or something, and then, go from there.

Traffic light detection and stop sign detection, red, yellow, green.

And I think you can, you could, I mean, if you wanted to do a full, I was just trying to make something that would stay in between the lanes on a highway, but if you wanted to do the full, the full, you know, set of capabilities needed for a driverless car, I think you could, and we'd done this at Cruise in the very first days, you can start off with a really simple human-written heuristic just to get the scaffolding in place for your system.

Traffic light detection, probably a really simple color thresholding on day 1 just to get the system up and running before you migrate to a deep learning based technique or something else.

And back when I was doing this, my first 1, it was on a Pentium 233 megahertz computer.

And I think I wrote the first version in basic, which is like an interpreted language.

Cause that's the thing I knew at the time.

And so there was no chance at all of using, there's no computational power to do any sort of reasonable deep nets like you have today.

So I don't know what kids these days are doing.

Are kids these days, you know, at age 13 using neural networks in their garage?

I get emails all the time from, you know, like 11, 12 year olds saying, I'm having, you know, I'm trying to follow this TensorFlow tutorial and I'm having this problem.

And the general approach in the deep learning community is of extreme optimism of, as opposed to, you mentioned like heuristics, you can separate the autonomous driving problem into modules and try to solve it rigorously, or you can just do it end to end.

And most people just kind of love the idea that us humans do it end to end, we just perceive and act.

We should be able to do the same kind of thing when you're on that.

And that kind of thinking, you don't want to criticize that kind of thinking because eventually they will be right.

And so it's exciting, and especially when they're younger, to explore that as a really exciting approach.

But yeah, it's changed the language, the kind of stuff you're tinkering with.

It's kind of exciting to see when these teenagers grow up.

Yeah, I can only imagine if your starting point is Python and TensorFlow at age 13, where you end up, you know, after 10 or 15 years of that, that's pretty cool.

Because of GitHub, because the state-of-the-art tools for solving most of the major problems in artificial intelligence are within a few lines of code for most kids, and that's incredible to think about, also on the entrepreneurial side.

And on that point, was there any thought about entrepreneurship before you came to college?

Is sort of doing, you're building this into a thing that impacts the world on a large scale?

Yeah, I've always wanted to start a company.

I think that's, you know, just a cool concept of creating something and exchanging it for value or creating value, I guess.

So in high school, I was trying to build like, you know, servo motor drivers, little circuit boards and sell them online or other things like that.

And certainly knew at some point I wanted to do a startup, but it wasn't really, I'd say until college, until I felt like I had the, I guess the right combination of the environment, the smart people around you and some free time.

So you came to MIT as an undergrad, 2004.

And that's when the first DARPA Grand Challenge was happening.

The timing of that is beautifully poetic.

So how'd you get yourself involved in that 1?

And so a bunch of undergrads, myself included, started meeting and got together and tried to haggle together some sponsorships.

We got a vehicle donated, a bunch of sensors, and tried to put something together.

And so we had, our team was probably mostly freshmen and sophomores, you know, which was not really a fair fight against maybe the, you know, postdoc and faculty-led teams from other schools.

We had our vehicle drive by wire and, you know, very, very basic control and things.

But, on the day of the qualifying sort of prequalifying round, the 1 and only steering motor that we had purchased, the thing that we had retrofitted to turn the steering wheel on the truck, died.

And so our vehicle was just dead in the water, couldn't steer.

Was there, like how did your view of autonomous vehicles in terms of artificial intelligence evolve in this moment?

I mean, like you said, from the 80s there's been autonomous vehicles, but really that was the birth of the modern wave, the thing that captivated everyone's imagination that we can actually do this.

So, how, were you captivated in that way?

So how did your view of autonomous vehicles change at that point?

I'd say at that point in time it was a curiosity, as in like, is this really possible?

And I think that was generally the spirit and the purpose of that original DARPA Grand Challenge, which was to just get a whole bunch of really brilliant people exploring the space and pushing the limits.

And I think to this day that DARPA challenge with its million dollar prize pool was probably 1 of the most effective uses of taxpayer money, dollar for dollar, that I've seen.

Because that small initiative that DARPA put out, in my view, was the catalyst or the tipping point for this whole next wave of autonomous vehicle development.

So let me jump around a little bit on that point.

it was in the city, but it was very artificial and there's no pedestrians and there's very little human involvement except a few professional drivers.

And then there was the robotics challenge with humanoid robots.

So in your now role is looking at this, you're trying to solve 1 of the, autonomous driving, 1 of the harder, more difficult places in San Francisco.

Is there a role for DARPA to step in to also kind of help out, like challenge with new ideas, specifically like pedestrians and so on, all these kinds of interesting things?

Well, I haven't thought about it from that perspective.

Is there anything DARPA could do today to further accelerate things?

And I would say my instinct is that that's maybe not the highest and best use of their resources and time.

Because like kick-starting and spinning up the flywheel is I think what what they did in this case for very very little money.

But today this has become this has become like commercially interesting to very large companies and the amount of money going into it and the amount of people like going through your class and learning about these things and developing these skills is just you know orders of magnitude more than it was back then And so there's enough momentum and inertia and energy and investment dollars into this space right now that I don't, I think they're, I think they can just say mission accomplished and move on to the next area of technology that needs help.

So then stepping back to MIT, you left MIT during your junior year.

As I said, I always wanted to do a company or start a company and this opportunity landed in my lap, which was a couple of guys from Yale were starting a new company and I Googled them and found that they had started a company previously and sold it actually on eBay for about a quarter million bucks, which was a pretty interesting story.

So I thought to myself, these guys are, you know, rockstar entrepreneurs, they've done this before, they must be driving around in Ferraris because they sold their company.

And, you know, I thought I could learn a lot from them.

So I teamed up with those guys and, you know, went out during, Went out to California during IAP, which is MIT's month off, on a one-way ticket and basically never went back.

We were having so much fun, we felt like we were building something and creating something, and it was gonna be interesting that, you know, I was just all in and got completely hooked.

And that business was Justin TV, which is originally a reality show about a guy named Justin, which morphed into a live video streaming platform, which then morphed into what is Twitch today.

Looking back, it was just an obvious, I mean, 1 way ticket.

I mean, if we just pause on that for a second, there was no, How did you know these were the right guys?

You didn't think it was just follow the heart kind of thing?

Well, I didn't know, but you know, just trying something for a month during IAP seems pretty low risk, right?

And then, you know, well, maybe I'll take a semester off.

And then after 2 or 3 cycles of that, I eventually threw in the towel.

But I think it's, I guess in that case, I felt like I could always hit the undo button if I had to.

But nevertheless, from when you look in retrospect, I mean, it seems like a brave decision that it would be difficult for a lot of people to make.

I'd say that the general flux of people out of MIT at the time was mostly into finance or consulting jobs in Boston or New York.

And very few people were going to California to start companies.

But today I'd say that's probably inverted, which is just a sign of the times, I guess.

Yeah, So there's a story about midnight of March

18, 2007, where TechCrunch I guess announced Justin.TV earlier than it was supposed to a few hours.

I don't know if any of this is true, you can tell me.

And you and 1 of the folks at Justin TV, Emmett Shear, quoted through the night.

So let me say a few nice things that The article I read quoted Justin Kahn said that you were known for hero coding through problems and being a creative quote, creative genius.

So on that night, what was going through your head or maybe put another way, how do you solve these problems?

What's your approach to solving these kinds of problems where the line between success and failure seems to be pretty thin?

Well, first of all, that's nice of Justin to say that.

I think I would have been maybe 21 years old then and not very experienced at programming.

But as with everything in a startup, you're sort of racing against the clock.

And so our plan was the second we had this live streaming camera backpack up and running where Justin could wear it and no matter where he went in a city it would be streaming live video.

And so we thought we were there and the backpack was working and then we sent out all the emails to launch the company and do the press thing.

And then, we weren't quite actually there.

And then we thought, oh, well, they're not gonna announce it until maybe 10 a.m.

What is that like, you know, 17 hours to go?

Did you understand what could possibly, like how complicated was the system at that point?

So to get a live video feed that looked decent working from anywhere in San Francisco, I put together this system where we had like 3 or 4 cell phone data modems and They were like, we take the video stream and sort of spray it across these 3 or 4 modems and then try to catch all the packets on the other side, you know, with unreliable cell phone networks.

Yeah, and putting these like, you know, sort of protocols on top of all that to reassemble and reorder the packets and have time buffers and error correction and all that kind of stuff.

And the night before it was just staticky.

Every once in a while the image would go staticky and there would be this horrible like screeching audio noise because the audio was also corrupted.

And this would happen like every 5 to 10 minutes or so.

And it was a really off putting to the viewers.

What was the, you just freaking out behind a computer?

There's the word, are there other, other folks working on this problem?

Yeah, it's a little lonely because there's 4 of us working on the company and only 2 people really wrote code.

And Emmett wrote the website and the chat system and I wrote the software for this video streaming device and video server.

And so, you know, it was my sole responsibility to figure that out.

And I think it's those, you know, setting deadlines, trying to move quickly and everything, where you're in that moment of intense pressure that sometimes people do their best and most interesting work.

And so even though that was a terrible moment, I look back on it fondly, because that's like, you know, that's 1 of those character defining moments, I think.

So in 2013, October, you founded Cruise Automation.

So progressing forward, another exceptionally successful company was acquired by GM in 16 for $1 billion.

But in October of 2013, what was on your mind?

How does 1 seriously start to tackle 1 of the hardest robotics, most important impact for robotics problems of our age?

After going through Twitch, Twitch was, and is today, pretty successful.

But the work was, the result was entertainment mostly.

Like the better the product was, the more we would entertain people and then make money on the ad revenues and other things.

But I figured like, what is really the point of becoming a really good engineer and developing these skills other than my own enjoyment?

And I realized I wanted something that scratched more of an existential itch, like something that truly matters.

And so I basically made this list of requirements for a new, if I was gonna do another company.

And the 1 thing I knew in the back of my head that Twitch took like 8 years to become successful.

And so whatever I do, I better be willing to commit, you know, at least 10 years to something.

And when you think about things from that perspective, you certainly, I think, raise the bar on what you choose to work on.

So for me, the 3 things were, it had to be something where the technology itself determines the success of the product, like hard, really juicy technology problems, because that's what motivates me.

And then it had to have a direct and positive impact on society in some way.

So an example would be like, you know, healthcare, self-driving cars, because they save lives, other things where there's a clear connection to somehow improving other people's lives.

And the last 1 is it had to be a big business because for the positive impact to matter, it's gotta be a large

And I was thinking about that for a while and I made like, I tried writing a Gmail clone and looked at some other ideas.

And then it just sort of light bulb went off like self-driving cars.

Like that was the most fun I had ever had in college working on that.

And like, well, what's the state of the technology has been 10 years, maybe, maybe times have changed and maybe now is the time to make this work.

And I poked around and looked at the only other thing out there really at the time was the Google self-driving car project.

And I thought, surely there's a way to, have an entrepreneur mindset and sort of solve the minimum viable product here.

And so I just took the plunge right then and there and said, this is something I know I can commit 10 years to.

It's probably the greatest applied AI problem of our generation.

And if it works, it's gonna be both a huge business and therefore like probably the most positive impact I can possibly have on the world.

So after that light bulb went off, I went all in on cruise immediately and got to work.

Did you have an idea how to solve this problem?

You know, slow, like we just had Oliver from Voyage here, slow moving retirement communities, urban driving, highway driving.

Did you have, like, did you have a vision of the city of the future or you know the transportation is largely automated that kind of thing or was it sort of more fuzzy and gray area than that?

My analysis of the situation is that Google's putting a lot had been putting a lot of money into that project.

They had a lot more resources and so, and they still hadn't cracked the fully driverless car, you know, this is 20, 2013, I guess.

So I thought, what, what can I do to sort of go from 0 to significant scale so I can actually solve the real problem, which is the driverless cars?

See all Lex Fridman transcripts on Youtube

Kyle Vogt: Cruise Automation | Lex Fridman Podcast #14