Dario Amodei is the Co-Founder and CEO of Anthropic. In the episode, we discuss detailed predictions on the AI industry for 2024, 2025, and beyond. Dario discusses his days at OpenAI, leaving to start Anthropic, why he doesn’t like the term AGI, what AI developments he’s most excited about right now and much more. Overall, an enlightening conversation for anyone interested in the future of AI. 

For more longform content with leading voices in AI, check out Redpoint's AI podcast, Unsupervised Learning: https://open.spotify.com/show/1ILHVeQ1jbI3pClQiy8oN6?si=05427b08875345db

(0:00) Intro
(1:20) Joining OpenAI
(14:31) Are scaling and AI safety intertwined?
(20:31) Anthropic Early Days
(24:04) Amazon's Investment in Anthropic
(24:19) FTX investment in Anthropic
(25:50) Anthropic's Business Today
(30:51) Dario's Advice For Builders
(34:26) Should we pause AI progress? 
(36:27) Future of AI
(37:55) Dario's Biggest AI Safety Concerns
(44:57) How Anthropic Deals With AI Bias
(49:29) Anthropic's Responsible Scaling Policy
(56:36) Testifying in front of Congress
(59:25) Will AI destroy humanity?
(1:00:00) GPT3 vs GPT4
(1:02:26) The memification of a CEO
(1:09:30) What are you most surprised by with AI?
(1:17:03) Why don't you like the term AGI?
(1:21:50) 2024 AI Predictions
(1:33:45) Dario's opinion on open-source models
(1:38:03) Probability of AI Catastrophe
(1:40:52) Misuse of AI
(1:44:44) Looking ahead: Dario's optimistic outlook on AI

Mixed and edited: Justin Hrabovsky
Produced: Rashad Assir
Executive Producer: Josh Machiz
Music: Griff Lawson

🎙 Listen to the show
Apple Podcasts: https://podcasts.apple.com/us/podcast/the-logan-bartlett-show/id1606770839
Spotify: https://open.spotify.com/show/5WqBqDb4br3LlyVrdqOYYb?si=3076e6c1b5c94d63&nd=1
Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS9zb0hJZkhWbg
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1


Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
🐦 Twitter - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow


About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode, Logan goes behind the scenes with world-class entrepreneurs and investors. If you're interested in the real inside baseball of tech, entrepreneurship, and start-up investing, tune in every Friday for new episodes.

I really freaked out about this stuff in 2018 or 2019 or so.

I didn't know if I was crazy or prescient.

On this episode, what you're going to hear is a conversation I had with CEO and co-founder of Anthropic, Dario Amadei.

We talk about a bunch of different things including predictions for the future of artificial intelligence, his upbringing, how he came to found Anthropic and spin out from open AI, as well as what problems he thinks can be solved by artificial intelligence in the near future.

Really interesting conversation with a very thoughtful person in an important position of power running 1 of the leading AI companies in the world.

If we can avoid the downsides, then the stuff about curing cancer, extending the human lifespan, solving problems like mental illness, this all sounds utopian, but I don't think it's outside the scope of what the technology can do.

So, chance that something goes quite catastrophically wrong on the scale of human civilization, you know, might be somewhere between 10 and

If you're enjoying this discussion and conversation with Dario, please do subscribe to this podcast on whatever channel you're listening to.

And if you're here to listen to more topics of artificial intelligence, I would encourage you to subscribe to Red Point's podcast on the subject, Unsupervised Learning, which you can find on any podcast player now.

So I normally don't tell people's backgrounds in a linear fashion, but I actually haven't heard yours.

I don't know if you've ever really told it in earnest, like childhood growing up, sort of what led you to starting Anthropic.

So maybe can you share a little bit about like your background, childhood growing up?

So I don't know that my childhood was that interesting or that that different from, you know, from people who are in tech or found companies.

I mean, I was always really interested in math.

It felt like it had a sense of objectivity, right?

You know, 1 kid could say, oh, this show is great.

And the other kid could say, oh, it's terrible.

But you know, when you're doing math, you're like, oh, man, there's an objective answer to this.

So that was always very interesting to me.

And I grew up with a younger sister who's 1 of my co-founders, and we always wanted to save the world together.

So it's actually kind of amusing that we're working on something together that at least potentially could have very wide scope.

So, yeah, I mean, in terms of how I got from there to Anthropic, my interest in math led me to be, you know, physics major undergrad.

But near the end of undergrad, I started reading, it was initially the work of Ray Kurzweil, who I think is a bit crazy about a lot of things, but just the basic idea that there's this exponential acceleration of compute, and that that's going to provide us enough compute.

Somehow, we had no idea how then, we had no idea it was going to be neural nets, will somehow get us to like very powerful AI.

So I was about to start grad school for theoretical physics and decided as soon as I got there that I wanted to do biophysics and computational neuroscience because if there was going to be AI, it didn't feel like AI was working yet.

And so I wanted to study the closest thing to that that there was, which was our brains.

You know, the closest, it's a natural intelligence, so therefore the closest thing to an artificial intelligence that exists.

So I studied that for a few years and kind of worked on networks of real neurons.

And then, you know, shortly after I graduated, I was at Stanford for a bit.

And then I saw a lot of the work coming out of Google, of Andrew Ng's group at Stanford.

And so I said, okay, you know, I should I should get involved in this area You know my reaction at the time was oh my god.

I'm so late to this area The revolution has already happened in this way years this year 2014, right?

So so, you know, I was just like, oh my god, like, you know this tiny community of 50 people like there are the Giants of This field it's too late to get in if I rush in maybe I can get some of the scraps.

That was my mentality when I entered the field.

Now, of course, it's 9 years later than that.

I interview someone every day who's like, I really want to get into this field.

And so I ended up working with Andrew Ng at Baidu for a year.

I ended up working at Google Brain for a year.

Then I was 1 of the first people to join OpenAI in 2016.

I was there for about 5 years and by the end of it I was VP of Research, was driving a lot of the research agenda.

We built GPT-2, GPT-3, reinforcement learning from human feedback, which is of course the method that's used in chat GPT and used along with other methods in our model Claude.

And 1 of the big themes of those 5 years was this idea of scaling, that you can put more data, more compute into the AI models, and they just get better and better.

The second thesis that was central is you don't get everything that way.

You can scale the models up, but there are questions that are unanswered.

It's ultimately sort of the fact value distinction.

You scale the model up, it learns more and more about the world, but you're not telling it how to act, how to behave, what goals to pursue.

And so that dangling thread, that 3 variable was the second thing.

And so those were really the 2 lessons that I learned.

And of course, those ended up being the 2 things that Anthropic is really about.

Was it just that this seemed to be where the smart people were going?

Yeah, so I was actually initially invited to join the organization before it existed in late 2015 as it was forming and decided not to.

But then a few months after it started, a bunch of smart people ended up joining.

And so I said, oh, maybe I'll do this after all.

You were there for a number of years, and at some point you made the decision that Anthropic was gonna be, I guess it wasn't initially, you've had an unusual path, it wasn't initially a company, right?

Originally it was started as a quasi-research lab, is that fair?

I mean, our strategy has certainly evolved, but actually it was a for-profit public benefit corporation since the beginning.

And even since the beginning, we had something on our website saying, we're doing research for now, but we see the potential for commercial activity down the road.

So I think all of these things we kind of kept open as potentialities, but you know, you're right.

For the first year and a half, it was mostly building technology and we were agnostic on, you know, what exactly we were gonna do with that technology or when.

We kind of wanted to keep our options open.

We felt it was better than saying, we're about this or we're about that.

And what was the thought at the time of, hey, we should go do something on our own.

Yeah, it was definitely the thoughts of a group of people.

And then I think in total, we got 14 or 15 people from OpenAI, which was about 10% of the size of the organization at the time.

It's funny to look back on those days because in those days, we were the language model part of OpenAI.

We, along with a couple of people who stayed, were those who had developed and scaled the language models.

There were many different other things going on at OpenAI, right?

There was, you know, there was a robotics project, a theorem proving project, projects to play video games.

But, you know, we felt that we were this kind of coherent group of people.

We had this view about language models and scaling, which to be fair, I think the organization supported.

But then we also had this view about we need to make these models safe in a certain way and, you know, we need to do them within an organization where we can really believe that these principles are incorporated top to bottom.

Open AI had a whole bunch of different things and still does experimenting around.

Like, was it evident at what point along the way was it evident that large language models were something that there was a lot of wood to chop and a lot of opportunity around?

Yeah, I mean, I think, I don't know, it was obvious at different times to different people.

So, you know, for me, I think there were a couple things.

You know, the general scaling hypothesis.

I wrote this document called the Big Blob of Compute in 2017, which I'll, you know, probably publish at some point, although it's primarily of historical interest now.

And so very much in my mind, and I think the minds of a small number of other people on both the team that left and the team that didn't leave, and some in other places in the world as well, it was clear that there was really something to scaling.

And then as soon as I saw GPT-1, which was done by Alec Radford, who's still at OpenAI, Our team actually had nothing to do with GPT-1, but we recognized it immediately and saw that the right thing to do with it was to scale it.

And so for me, everything was clear at that moment.

And even more clear as we kind of scaled up to GPT-2 when we saw that the model was capable of, my favorite thing was like, we were able to get the model to perform a regression analysis.

So you'd give it like the price of a house and ask it to predict number of square feet or something like that.

You gave it a bunch of examples, then you gave it 1 more price and you're like, how many square feet?

It didn't do great, but it did better than random.

And in those days, I'm like, oh my God, this is like some kind of general reasoning and prediction engine.

Like, oh my God, what is this thing I have in my hands, right?

So, you know, it has been my view ever since then that, you know, this would be not just language models, but, you know, language models is exemplar of the kinds of things you can scale up.

You know, this would be, you know, really, really central to the future of technology.

Did you consider yourself like a founder or a CEO prior to actually doing it?

No, so I really kind of never thought of myself that way.

Like if you went back to, you know, me in childhood, it would have been very unsurprising, you know, for me to be a scientist.

But you know, I never kind of thought of myself as like a founder or a CEO or a business person, right?

You know, I always thought of myself as a scientist, someone who discovers things.

But I think just having been at several different organizations convinced me that in fact, I did have my own vision of how to run a company or how to run an organization because I'd seen so many and I thought, well, I don't know I'd actually do it this way, right?

And so sort of the contrast, you know, not that I disagreed with every decision that made, but just watching all these decisions go by took me to the point where I'm like, actually, I do have opinions on these questions, right?

I do have an idea of how you would grow an organization, how you would run a research effort, how you would bring the products of that research organization out into the world in a way that makes business sense, but is also responsible.

I don't think I really had those thoughts naturally, but as I was brought into contact with organizations that did that, then I became excited about those things.

I kind of almost reluctantly learned that I actually had strong opinions on these things.

Not to draw it in contrast to any specific names, but maybe just in others in the field that I would consider large language foundation model as a product.

What's something foundational, I guess not to use a cute term, but something foundational that you believe at Anthropic, that you would draw in distinction to others in the space.

So, you know, 1 is just this idea that we should be building in safety from the beginning.

Now, I'm aware we're not the only ones who've said that, but I feel like we've built that, we've really built that in from the beginning.

We've thought about it from the beginning.

We've started from a place of caution and commercialized things, brought things out into the world, starting from, hey, you know, can we, can we open these switches 1 by 1 and see what actually makes sense?

I think in particular, you know, a way, a way I would think about it is that what we're aiming to do is not just to be successful as a company on our own, although we are trying to do that, but that we're also trying to kind of set a standard for the field, set the pace for the field.

So this is a concept we've called race to the top.

So race to the bottom is a popular term where everyone is competing to lower cost or delivers things as fast as possible.

And as a result, they cut corners and things get worse and worse.

And we always think about how not to contribute to it too much.

But there's also a concept of race to the top, which is that if you do something that looks better, it naturally has the effect that other players end up doing the same thing.

For a couple of years, we were the only org that worked on interpretability seen inside neural nets.

There are various corporate structures that we've implemented that we hope others may emulate.

And recently, we released this responsible scaling plan that I could talk more about later.

But generally, we're trying to set the pace.

We're trying to do something good, inspiring, also viable, and encourage others to do the same thing.

And at the end, again, maybe we win in a business sense.

But maybe someone else wins in a business sense.

But the thing that matters is that the standards are increasing.

I want to talk about all that stuff in a second.

But why do you think philosophically that AI development or the scaling of models and safety, why are they intertwined?

I've heard you maybe coiled together in different ways.

So I think this actually isn't such an unusual thing.

Building a bridge and making it safe, they aren't exactly the same thing.

But they both involve all the same principles of like civil engineering.

And it's kind of hard to work on bridge safety in the abstract aside from outside the context of like a concrete bridge.

Like what you need to do is you need to look at the bridge.

You're like, OK, well, these are the forces on it.

You know, these are the, this is the stress tensor.

This is, you know, the strength of the material or whatever.

That's the same thing you come up with in building the bridge.

If safety differs in any way, maybe it differs in thinking about the edge cases.

Like in safety, you have to worry what goes wrong 0.1 percent of the time.

Whereas in building the bridge, you have to think about the median case.

But it's all the same civil engineering, it's all the same forces of mechanical physics.

I think it's the same in AI and large language models.

In particular, safety is itself a task like, is this thing the model's doing right or wrong?

Where right or wrong could be something as prosaic as, is the model telling you how to hotwire a car, or as scary and sophisticated as, is the model going to help me build a bioweapon, or is it going to take over the world and make swarms of nanobots or whatever futuristic thing, figuring out whether the model is going to do that and the behavior of the model is itself an intellectual task of the kind that models do.

So the problem and the solution to the problem are mixed together in this way, where every time you get a more powerful model, you also gain the capability to understand and potentially reign in the models.

So we have this problem where these 2 things are just mixed together in a way that's I think hard to untangle.

I think the only reason that's surprising is that the community of people who thought about AI safety was historically very separate from the community of people who developed AI.

They came to it from a more philosophical perspective, more of a moral philosophy perspective, whereas those who built the technology were engineers.

But just because the communities were different, that doesn't imply that the actual content turned out to be different.

It was becoming a business and commercializing your effort.

If you could scale and figure all the safety stuff out without being a business?

Or is the business side of it inherently intertwined as well as something that interests you?

1 is I think it's actually going to be very difficult to build, or would have been very difficult to build models of the scale that we want without being a commercial enterprise.

I mean, people make jokes about VCs being willing to pour huge amounts of money into anything, but I think that's only true up to a point.

You guys have LPs, like things need to, it's not just all hype train, right?

And so, you know, we're now getting to the point where you need, you know, certainly multiple billions of dollars, and I think soon tens of billions of dollars to build models at the frontier.

And to study safety with those models, models at the frontier requires you to have intimate access to those models, particularly for tasks like interpretability.

So first of all, yeah, I just think it's very hard.

On the other hand, you know, or in support of that point, I also think that there are some things that you learn from the business side of things.

Some of it is just learning the muscle and the operation of things like trust and safety.

So, today we deal with trust and safety issues like, oh, people are trying to use a model for inappropriate purposes.

Not things that are going to end the world, but things that we'd rather that people not do.

I think the ultimate significance of being able to develop methods to address those things and kind of enforcing those things in practice when they're used at scale by users is it allows us to practice for the cases that are really, really high stakes.

I think without that organizational institutional practice, it might be difficult to just be thrown into the shark tank.

Congratulations, you've built this amazing model.

It can cure cancer, but also someone could make a bio-plague that would kill a million people.

You've never built a trust and safety org.

You have to deploy this model in the world and make sure we do 1 and not the other.

That would just be a very difficult thing to do, and I don't think we would get it right.

Now, all that said, I mean, I will freely admit my passion is the science and the safety.

The business stuff is quite a lot of fun.

I think we've just watched all the different customers.

Just learning about the whole business ecosystem has been great.

But definitely my first passion is the science of it and making sure it goes well.

Was there a serious debate about being a business versus not in the early days?

Was that a real conversation among the group?

Yeah, so I think certainly everyone was aware from the beginning that there was a good chance that we would commercialize the models at some point.

You know, we had this thing on our website.

I'm not sure if it's up there anymore, but you can see it on the way back machine that said, you know, for now we're doing research, but we see commercial potential down the road.

So everyone who joined saw that and everyone who joined knew that.

But there was a question of when exactly should we do it?

So there was a period around, I think it was April, May, June of 2022 when we had the first version of CLAWD, which was actually like a smaller model than CLAWD1, but we were training the model that would become CLAWD1 at that time.

We realized that with RL from human feedback, we didn't have our constitutional AI method yet, that This thing was actually great to interact with, and all of our employees were having fun interacting with it on Slack.

We showed it to a small number of external people, and they had lots of fun interacting with it.

So it definitely occurred to me and others that, hey, there could have been a lot of commercial potential to this.

I don't think we anticipated the explosion that happened at the end of the year.

I don't think we saw that much potential.

But yeah, we, you know, we definitely had a debate about it and I wasn't sure quite what to do.

I think our concern was that with the rate at which the technology was progressing, a kind of big, loud public release might accelerate things so fast that the ecosystem might not know how to handle it.

And, you know, I didn't want our kind of first act on the public stage, you know, after we'd set it, you know, after we put so much effort into being responsible to accelerate things, to accelerate things so greatly.

I generally feel like we made the right call there.

You know, there's many, many pros, many cons, but I think overall, overall we made the right call.

And then, you know, certainly as, you know, as soon, as soon as the other models were out and kind of the gun had been fired, then we started putting these things out.

And indeed, we've managed to put ourselves among the top 2 or 3 players in this space.

Was that gun being fired and chat GPT sort of taking off, was that similar to the maybe fear that you had and of like, hey, this might start a race?

So, you know, I think we saw it with, you know, Google's, you know, Google's reaction to it, you know, that there was definitely, you know, just judging from the public statements, a sense of fear and existential threat.

And I think they responded in a very economically rational way.

But you put the 2 things together and it really created an environment where things were racing forward very quickly.

And look, I love technology as much as the next person.

There was something like super exciting about the whole, make them dance, oh, we're responding with something.

I mean, I can get just as excited about this as everyone.

But given the rate at which the technology is progressing, you know, there was a worrying aspect about this as well.

And so in this case, I'm at least at least on balance, glad that, you know, we weren't the ones who fired that starting gun.

Well, you recently announced an investment from Amazon.

Before that, you did a round with Spark and a little bit more traditional venture capitalists.

The round with Amazon, it's complicated and I can't go into the details, but it's not a full clothes shirt, priced for

Before that, you had an unusual round as well with FTX, right?

See all The Logan Bartlett Show transcripts on Youtube

Anthropic CEO on Leaving OpenAI and Predictions for Future of AI