This is a talk by Marc Raibert for course 6.S099: Artificial General Intelligence. He is the CEO of Boston Dynamics. This class is free and open to everyone. Our goal is to take an engineering approach to exploring possible paths toward building human-level intelligence for a better world.

Note: Due to technical difficulties, we don't have a screencast of the slides, and the video of the slides is low resolution. Despite this, I chose to include several parts of the talk that show slides, especially with videos. It's not optimal, but I hope you learn and enjoy anyway. Thanks for understanding. We're always learning and improving.

Note 2: Marc and I asked people not to release the videos around the 20 minute mark before they are officially released by Boston Dynamics. They have been now, go check them out: https://www.youtube.com/user/BostonDynamics

OUTLINE:
0:00 - Introduction
1:06 - Slides
24:27 - Demo
33:40 - Q&A

INFO:
Course website: https://agi.mit.edu
Contact: agi@mit.edu
Playlist: http://bit.ly/2EcbaKf

CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- AI Podcast: https://lexfridman.com/ai/
- Show your support: https://www.patreon.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Twitter: https://twitter.com/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Slack: https://deep-mit-slack.herokuapp.com

Welcome back to 6S099, Artificial General Intelligence.

He really doesn't need an introduction, but we'll give him 1 anyway.

He's the founder and CEO of Boston Dynamics.

He founded the CMU Leg Lab in 1980, the MIT Leg Lab in 1986, Boston Dynamics in 1992.

He and his team have developed some of the most amazing robots ever built, including Big Dog, Atlas, Handle, Spot, Spot Mini.

These robots move with the agility, dexterity, and even grace that rivals and often supersedes that of human movement.

He continues to inspire us with what robots are capable of achieving in the real world and what physical form future intelligence systems may take as they become integrated in our daily lives.

This is our grand mission, our aspiration, which is to make robots that are equal to or greater than people and animals.

And it's a daunting mission because we're so good at things.

It seems effortless, I'm standing here, no 1 questions that I could stand here like this, but a lot's going on.

I can manipulate things, I can pick up this water, or I can reach in my pocket and use my hands with all the sensors in my hands and coordinate that.

And maybe most of all, our perception systems.

You know, this audience has, what is it, 250 people in it or something?

And I can look out there and see every 1 of you stabilized in space, even while I'm moving.

And our goal is to keep chipping away to try and get there.

Before I get started, I wanted to say that I got my start in robotics here at MIT.

I was in the, what was then called the psychology department, the brain and cognitive science department, but I was taking an IAP course, just like you are, and it might have been exactly this time of year, when I followed my professor, Bertolt Horn, back, I was jabbering away at him, asking him some questions about this or that, and we walked back to Tech Square, which is where the AI lab was in those days, and we went up to the ninth floor, and Russell Novsker, who was a guy working in the lab, had an arm all taken apart on the table.

I didn't switch my major but I got Bertolt to be an advisor.

I found a topic that had to do with robotics.

In that time it was a manipulation thing, but eventually became a legged thing.

And it was amazing, you know, and I've never looked back.

So here are some animals doing things that are very exciting, climbing around on very rough terrain, very short footed, using a mixture of their proprioception and their vision.

And look, there's even a baby that probably is only a couple of months old, has no trouble at all doing these things.

And look at the grace and suppleness and the fearlessness of these animals.

Here are animals running for their lives.

The gazelle is trying to stay alive for the next 10 minutes, and the cheetah is trying to get a meal so that it can stay alive in general.

And you know, even people can do things that are breathtaking.

I assume all of you were out this morning getting a little exercise, climbing up the green building that we're in now and maybe the other places around here.

It's funny, I bumped into some people when I came into the room who were climbing the stairways, I think they were going on a trek up and down them, but I'd like to see the going outside.

So probably most of you have seen this video.

This is sort of where we were after about 10 years of work attempting to make machines that could work out in the real world that were dynamically stabilized.

Dynamics is a big deal for our company and for what we do.

So some active sensing and control and understanding of the physics.

Here, this robot has all its control on board and it has reflexes and sensors.

And this is an extension to a 1,000 pound robot that could carry about 400 pounds of payload.

And we took it all around the United States, testing it in various situations.

Here we have it in Virginia doing some bushwhacking.

It's actually following a person, but the person is only in and out of view intermittently, so it has to be able to keep track of where the person is and deal with that.

And then back in good old Boston, 10 inches of snow, just marches right up the hill.

People know Songbei, who's doing the MIT cheetah.

And basically an experiment in seeing how fast we could make something like this run.

Although you notice it's on a parking lot so it wasn't doing this on rough terrain and getting both the efficiency and the speed in the context of a machine that also can do rough terrain is a really big challenge that remains with us.

So this is just a snapshot of most of the robots we've built at Boston Dynamics over the years.

And I'm not gonna talk about most of them, I'm just gonna talk about the last 4.

These are all robots that we developed since we've been part of Google, which has been the last 4 years.

Spot Mini, there's a Spot Mini on the floor here, which we'll demo a little later.

Some of you may remember the Humanoid that we used in the DARPA Robotics Challenge.

And then Handle, which is our latest version.

So I'll have a few words to say about each of them.

So we had been developing Big Dog and those other quadrupeds that I showed you for quite some number of years.

And it was amazing for us to find out when we did this project on Spot, this is the predecessor to that, that there was still a lot to learn.

And we kind of revolutionized the hardware design and how the control worked and got a much higher level of rough terrain performance.

And part of the solution to that was to be able to decompose the control problem into many separate controllers that operated in different regions of state space.

And that allowed us both to have programmers work on multiple solutions to the problem and also have the complexity of each controller simplified by only having to operate in a small part of the dynamic space.

Here we've added a robot arm to the previous version of Spot.

And we believe that mobile manipulation, that is manipulation when you can move the base, is really a powerful way of doing things.

Now this is probably the most important thing I wanna show tonight, and I'll show it 3 different times.

The idea that we don't build controllers that just do 1 particular thing, but that they can determine where they are in the execution, here's another version of it, and then adjust what they're doing in order to compensate for disturbances in the real world.

I know this class is about AI and probably autonomy.

I think that 1 of the most important ways of getting to autonomy is to have the low level implementations very robust to disturbances so that the planning steps don't have to take care of all the minutia of the details of the real world.

And that's what we've been trying to do there.

We've been experimenting with doing delivery of packages to people's houses.

These are all employees of Boston Dynamics so we didn't go crashing ordinary people's houses.

And it turns out that there's just lots of different kinds of stairways and entranceways and the robot's doing very well, We're up to something between 70 and 80% of the kinds of stairs and access places we encounter after collecting data and making improvements and adjustments.

So I'm gonna say a few philosophical things or approach things.

A lot of people think that this is the model of how a computer and a robot interact.

That is, there's the robot, which is hardware and electronics and sensors, and then there's a computer, and that the computer listens to the sensors on the robot and then gives it instructions and tells it what to do.

And while I think that's actually going on, there's another part to the story, which is that the physical world is also giving instructions to the robot.

And that means that the energies stored in the robot, either in its springs or in its motion, those are all important determinants of how the robot's gonna behave in the time coming forward.

And so we like to think in terms of designing the hardware of the robot, the physical world, and the computer all as 1 holistic thing where we take into account those interactions.

You know, a harmonic system is 1 usually where you have energy oscillating back and forth.

Almost all legged locomotion has some amount of harmony going on between potential energy of elevation, potential energy of elastic deformation, kinetic energy of motion, and inverted pendulum things and the like.

Another part of our approach we call build it, break it, fix it.

Now I have friends who build their robots and are so into the beauty of what they've created that they kind of put it on an altar and afraid of actually hurting it.

So in fact I even have friends here at MIT that have done that, where they have a gold-plated robot and they're afraid of taking it out into the world.

Every 1 of our robots is designed to get bashed to bits.

We have staff who are there to fix the robot on a daily basis as we break it.

And I think doing that, build it, break it, fix it, means that we're able to learn a lot from the actual physical robot working in the world.

And we can use that knowledge in order to improve the robot, improve its behavior, and we really like to go around that loop as quickly as we can, early in the process, and do it as many times as we can.

So here's what Build it, Break it, Fix it looks like.

Now this robot's supposed to be using its visual system to avoid the trees.

I think it might have fallen in love with this tree.

We don't purposely give them any emotion, but boy it's hard not to see that.

And here's the first time we tested the push response to this robot.

So some guy who just started that week, Trent, had $5,000 which we paid for in repairs to his vintage BMW.

So the last thing sort of about philosophy is long-term versus short-term.

You know, our company is 25 years old and we've mostly been a long-term robotics company.

That is we're interested in moving the boundary forward in what robots can do and we're interested in you know making it so robots meet the dream of being the equal or better than people and animals.

But lately we've started to realize that some of our robots have enough capability that maybe it's time to try and productize them and we will learn a lot by doing that too.

1 of the things for instance that I've always claimed is that we always spent a lot of money on building our robots and use that as a competitive advantage.

That is, DARPA was a frequent funder of us.

DARPA always said, let's take money out of the equation and just figure out how to get the solution and then worry about getting the cost down later.

So I always assumed and argued that once we get a robot doing things that are interesting then you can go and redesign it to make it lower cost.

Well we're going to test that because it might not be true.

It might be that we've designed ourselves into an expensive corner and that it might be too late.

But the robot that we'll show in a little bit is much significantly cost reduced from the prototype of it and it'll be interesting to see whether we can get it down to the kind of prices that are useful.

So this is just a picture again of the idea of aiming long but also aiming short.

And I think it's gonna be a challenge to see whether we can keep the culture of the company to support both of these directions.

Because people manufacturing stuff have a different mindset than people trying to get out to the future horizons.

And it's gonna be a challenge to keep both those kinds of people happy.

Here's some of the things that, some of the kinds of applications you can look at based on modest technical capabilities.

I've shown mobility and manipulation here, but you could put cost, reliability, there's many things that could be on these axes.

You know, entertainment like robots and theme parks is something that I think we should be able to do.

I think home delivery is waiting for self-driving cars to get all the way there, self-driving trucks, and once they do, then we will be working on getting it from the truck to the home.

Logistics, There's about a trillion boxes moved every year around the world and most of it's done by hand and so there's really a big opportunity to having robots help with moving those trillion boxes.

Security which could mean either commercial security like patrolling your shopping center or the military type security, construction.

A lot of people have been coming to us with their construction applications asking if we can help.

And you know I'm not going to talk about it now but if afterwards you want to ask about that I can fill you in a little more.

And I think this is really the ultimate home run application, care for the elderly and the disabled.

I used to say that I wanted to have robots that would help me take care of my parents and older people, but I realize now that it's probably going to be my children using them to help take care of me, but you guys are all a little bit younger and I think there'll be a time when you could use robots to help make your parents' lives better.

Now some of you may think that your parents don't want that, but I think it's a complex question.

We've seen some surveys that say that, people aren't totally happy with the idea of their kids taking care of them on a moment by moment basis.

And I think there's gonna be an opportunity for doing something.

But technically, this is still a ways off.

Spot Mini is a robot that weighs about 60 pounds.

That previous spot weighed about 180 pounds.

It's got an arm with 5 degrees of freedom.

Batteries for these things are a challenge because you can have consumer products like electric drills that have relatively small batteries and then there's electric cars that have big batteries and there's not really much available in between.

So we've done a lot of work on the battery technology for these things to make them safe and reliable and hot swappable and things like that.

The previous version had 3 quad core i7s, this 1 has 2.

And then there can be some sensors, LIDARs, stereo, and the like.

So you can see Spot Mini is a little bit smaller than Spot.

This isn't a real house, and those aren't real people, those are engineers.

This is inside of a warehouse we have out on 128 where we've built a house.

You can see that we don't mind scuffing up the walls here and there is a lot of scuffing that happens.

Some of you may recognize Zach Tuchowski who's an MIT alum and he's again disturbing the robot.

Here the robot's using its vision to do some stepping stone type operations.

And I think Jean is gonna talk a little bit more about this in a couple of minutes.

And here's a case where it's doing stepping stones on real stones.

And it's keeping its balance, figuring out where to put the feet.

And again, this robot only has stereo looking out the front, whereas this 1 has stereo on all 4 sides.

Now 1 of the cool things about animals is that they have these stabilization mechanisms for their sensors.

And here's our attempt to show that this robot can do the same sort of thing.

And if you think about it, when you're manipulating, you really want the hand to be stabilized in space, and so you'd like the body to be able to kind of coordinate with the hand so that you can concentrate on what the real world task is.

Oh man, you guys didn't pick up the banana peels, huh?

So our concept for the Spot Mini product is to make a platform.

It's sort of the, we're thinking of it like the Android of robots.

So with Android, there's a hardware platform, and then there's a software platform, and then developers, third-party developers, create their own apps that use the platform.

So we've made this spot so that there's a place to mount hardware on the robot, but there's also an API to program it through, and then there's a facility to have additional computing external to the robot, and we're working with third parties to develop their own applications that run on the platform.

This is a video that we haven't been able to release publicly.

Please don't tape it and show it because I'll explain later if you want to know why not, but this is just revealing that we do have an arm on the new version of Spot.

It's using a camera in the hand to find the door handle.

This robot doesn't weigh a lot, so it has to use tricks to keep the door open, so that's why it puts its foot in the door.

And here again, we want to show that we've made the solution robust to certain kinds of disturbances.

So Andy there, Andy's sitting over here, is pushing on the door, pushing on the hand.

The robot keeps track of how far, how much progress it's made in doing its task.

It's so smart it even kicks that shell out of the way.

And now it's just gone back to try again.

Here the robot has, in a previous session, we've taken it around the lab, this is Boston Dynamics, taken it around the lab and recorded visual data that could be used for navigation.

And it's using its stereo to match up features in the environment so that it can navigate and go where it had gone in the previous path.

Every day around noon, the robot seems to show up and I hear it pausing out there.

Sometimes it comes up with a solution that isn't, in here you'll see another 1.

It comes up with a solution that isn't quite What you'd call as an optimization, but it does get a solution.

We call this patrol route, and we're working on developing a lot of software to support it to make it so that other people can capture a patrol route and then execute them on a routine basis and then do other tasks while they're on the patrol route.

So for this demo, Seth's got a joystick and he's telling it the speed to go in the forward direction and turning, but the robot's doing all its own gate selection, coordination of legs, balance obviously.

So the robot has a bunch of different gaits.

So here's trotting, which is diagonal pairs of legs.

It can do pacing, which is lateral pairs of legs working together.

I have to tell you, in the earliest days of me being involved in locomotion, I thought gait was a big deal.

And I don't think it's central to what matters, which is support, stability, propulsion, and things like that.

I just thought I'd say a couple of words about the mechanical side.

You know, Atlas is a new version of a humanoid.

I know Some of you worked with the DARPA Robotics Challenge humanoid, which was a big hulking thing that we made.

And the way we got there was to work on the elements of the mechanical design to take advantage of 3D printing and some optimization.

And we focused on 2 or 3 different things.

1 is making some of the leg parts where we embed hydraulic pathways, hydraulic actuators, places for valve mounts and filters and things like that into the leg.

There's a single upper leg part that incorporated about 15 or

different separate components in the previous design, which made it lighter, more compact, and higher strength to weight ratio.

We also developed a hydraulic power unit which takes many components.

The thing on the left are the components as separate ones, and we were able to print up parts that integrated them so that there was a motor, a pump inside of a motor, an accumulator, a reservoir, valves, filters, and those things, and we shrunk it down so that the robot could be smaller and lighter.

And using that approach we went from about a 375 pound DRC robot to a 190 pound robot and then the current 1 is about 165 pounds.

Now this picture might lead you to believe that I'm advertising myself as only weighing

And unfortunately that's not true, but I'm working on it.

And I don't know, I don't think we have this out as a video.

Here's some robot behavior that uses whole body motion, meaning the mobility base plus the arms plus the torso are all combining in order to handle these boxes.

It's using vision with the QR codes to simplify the task.

Here we're trying to go at human speeds of operation, and so the robot searches for a box using its vision.

I think that was the only take we ever got with both robots working together.

And 1 of the problems with YouTube is everybody's already seen what you've been up to by the time you go around to give a talk, so I imagine most of you have seen this.

But here's a parkour robot we're working on where we've actually strengthened the hips so that it can do a little bit more jumping and...

And it's kind of interesting that we've been interested in making a robot a little bit like the humanoid that has less degrees of freedom, fewer degrees of freedom and simpler.

And we designed this robot and the ultimate version of this will have about 10 joints whereas the humanoid had 28.

We have some use cases for this that I'm not gonna talk about today, but this robot can lift heavy loads.

It has a relatively small footprint given what its strength is.

So the way things are done in logistics now is to use big robot arms that take up a lot of floor area or heavy.

And we're looking at ways of using a robot like this 1.

Not exactly this, it's sort of an evolution of this design in order to do logistics operations.

Boston Dynamics is hiring, and I hope some of you will apply for a job there.

MIT alum that currently work at the company.

So I'm sort of making the point that these people are happy there, just like you could be.

And I hope you'll look at our website and see what we're looking for and consider it.

So I'm just gonna wrap up by talking about, you know, I used to be a professor here and at Carnegie Mellon.

And when I was a professor, we used to mostly wrote papers and we were excited by how many papers we could write and how many people cited them in their papers.

But as a company guy, instead of papers, I think we count YouTube hits.

And instead of citations, here I won't tell you what this is but most of you probably know.

So, now we count spoofs instead of citations.

And I'm happy to say that we're doing great.

And the upper left is in Akihabara, Japan.

The upper right is a Los Angeles online television show.

And I guess that's Appalachia on the right.

See all Lex Fridman transcripts on Youtube

Marc Raibert: Boston Dynamics | MIT Artificial Intelligence (AI)