Sam Altman is the CEO of OpenAI, the company behind GPT-4, ChatGPT, DALL-E, Codex, and many other state-of-the-art AI technologies. Please support this podcast by checking out our sponsors:
- NetSuite: http://netsuite.com/lex to get free product tour
- SimpliSafe: https://simplisafe.com/lex
- ExpressVPN: https://expressvpn.com/lexpod to get 3 months free

EPISODE LINKS:
Sam's Twitter: https://twitter.com/sama
OpenAI's Twitter: https://twitter.com/OpenAI
OpenAI's Website: https://openai.com
GPT-4 Website: https://openai.com/research/gpt-4

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
4:36 - GPT-4
16:02 - Political bias
23:03 - AI safety
43:43 - Neural network size
47:36 - AGI
1:09:05 - Fear
1:11:14 - Competition
1:13:33 - From non-profit to capped-profit
1:16:54 - Power
1:22:06 - Elon Musk
1:30:32 - Political pressure
1:48:46 - Truth and misinformation
2:01:09 - Microsoft
2:05:09 - SVB bank collapse
2:10:00 - Anthropomorphism
2:14:03 - Future applications
2:17:54 - Advice for young people
2:20:33 - Meaning of life

SOCIAL:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Reddit: https://reddit.com/r/lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

We have been a misunderstood and badly mocked org for a long time.

Like when we started, we like announced the org at the end of 2015 and said we were going to work on AGI, like people thought we were batshit insane.

You know, like I remember at the time a eminent AI scientist at a large industrial AI lab was DMing individual reporters being like, these people aren't very good and it's ridiculous to talk about AGI and I can't believe you're giving them time of day.

And it's like, that was the level of like pettiness and rancor in the field at a new group of people saying, we're gonna try to build AGI.

So OpenAI and DeepMind was a small collection of folks who are brave enough to talk about AGI in the face of mockery.

The following is a conversation with Sam Altman, CEO of OpenAI, the company behind GPT-4, JAD-GPT, DALI, Codex, and many other AI technologies, which both individually and together constitute some of the greatest breakthroughs in the history of artificial intelligence, computing,

a few words about the possibilities and the dangers of AI in this current moment in the history of human civilization.

We stand on the precipice of fundamental societal transformation where soon, nobody knows when, but many including me believe it's within our lifetime.

The collective intelligence of the human species begins to pale in comparison by many orders of magnitude to the general super intelligence in the AI systems we build and deploy at scale.

It is exciting because of the innumerable applications we know and don't yet know that will empower humans to create, to flourish, to escape the widespread poverty and suffering that exists in the world today, and to succeed in that old, all too human pursuit of happiness.

It is terrifying because of the power that super-intelligent AGI wields to destroy human civilization, intentionally or unintentionally.

The power to suffocate the human spirit in the totalitarian way of George Orwell's 1984, or the pleasure-fueled mass hysteria of Brave New World, where, as Huxley saw it, people come to love their oppression, to adore the technologies that undo their capacities to think.

That is why these conversations with the leaders, engineers, and philosophers, both optimists and cynics, is important now.

These are not merely technical conversations about AI.

These are conversations about power, about companies, institutions, and political systems that deploy, check, and balance this power, about distributed economic systems that incentivize the safety and human alignment of this power.

About the psychology of the engineers and leaders that deploy AGI and about the history of human nature, our capacity for good and evil at scale.

I'm deeply honored to have gotten to know and to have spoken with on and off the mic with many folks who now work at OpenAI, including Sam Altman, Greg Brockman, Ilya Sutskever, Wojciech Zaremba, Andrej Karpathy, Jakub Pachacki and many others.

It means the world that Sam has been totally open with me, willing to have multiple conversations, including challenging ones, on and off the mic.

I will continue to have these conversations to both celebrate the incredible accomplishments of the AI community and to steel man the critical perspective on major decisions various companies and leaders make, always with the goal of trying to help in my small way.

To support it, Please check out our sponsors in the description.

And now, dear friends, here's Sam Altman.

It's a system that we'll look back at and say it was a very early AI.

And it's slow, it's buggy, it doesn't do a lot of things very well, but neither did the very earliest computers.

And they still pointed a path to something that was gonna be really important in our lives, even though it took a few decades to evolve.

Like out of all the versions of GPT 50 years from now, when they look back on an early system, that was really kind of a leap.

You know, in a Wikipedia page about the history of artificial intelligence, which of the GPTs would they put?

I sort of think of progress as this continual exponential.

It's not like we could say here was the moment where AI went from not happening to happening.

And I'd have a very hard time like pinpointing a single thing.

Will the history books write about GPT-1 or 2 or 3 or 4 or 7?

That's for them to decide, I don't really know.

I think If I had to pick some moment from what we've seen so far, I'd sort of pick ChatGPT.

You know, it wasn't the underlying model that mattered, it was the usability of it, both the RLHF and the interface to it.

Reinforcement learning with human feedback.

What was that little magic ingredient to the dish that made it so much more delicious?

So we train these models on a lot of text data and in that process they learn the underlying something about the underlying representations of what's in here or in there and they can do amazing things.

But when you first play with that base model that we call it after you finish training It can do very well on evals, it can pass tests, it can do a lot of, you know, there's knowledge in there.

But it's not very useful, or at least it's not easy to use, let's say.

And RLHF is how we take some human feedback.

The simplest version of this is show 2 outputs, ask which 1 is better than the other, which 1 the human raters prefer, and then feed that back into the model with reinforcement learning.

And that process works remarkably well with, in my opinion, remarkably little data to make the model more useful.

So RLHF is how we align the model to what humans want it to do.

So there's a giant language model that's trained on a giant data set to create this kind of background wisdom knowledge that's contained within the internet.

And then somehow adding a little bit of human guidance on top of

it through this process makes it seem so much more awesome.

Maybe just because it's much easier to use.

You get it right more often the first time and ease of use matters a lot even if the base capability was there before.

And like a feeling like it understood the question you're asking or like it feels like you're kind of on the same page.

I mean, that could be a more technical term for it.

And you're saying that not much data is required for that, not much human supervision is required for that.

To be fair, we understand the science of this part at a much earlier stage than we do the science of creating these large pre-trained models in the first place, but yes, less data, much less data.

That's so interesting, the science of human guidance.

That's a very interesting science, and it's going to be a very important science to understand how to make it usable, how to make it wise, how to make it ethical, how to make it aligned in terms of all the kind of stuff we think about.

And it matters which are the humans and what is the process of incorporating that human feedback and what are you asking the humans?

What aspects are you letting or asking the humans to focus in on.

But what is the data set it's trained on?

Can you kind of loosely speak to the enormity of this data set?

We spend a huge amount of effort pulling that together from many different sources.

There's like a lot of, there are open source databases of information.

So some of it is Reddit, some of it is news sources, all like a huge number of newspapers.

the world, more than I think most people think.

Like where the task is not to find stuff but to filter out

Because there seems to be several components to solve the design of the, you could say, algorithms, so like the architecture of the neural networks, maybe the size of the neural network.

There's the human supervised aspect of it, with RL with human feedback.

is not that well understood about creation of this final product, like what it takes to make GPT-4, the version of it we actually ship out and that you get to use inside of ChatGPT, the number of pieces that have to all come together, and then we have to figure out either new ideas or just execute existing ideas really well at every stage of this pipeline,

You've already said for GPT-4 in the blog post and in general, there's already kind of a maturity that's happening on some of these steps.

Like being able to predict before doing the full training of how the model will behave.

That there's like a law of science that lets you predict, for these inputs, here's what's gonna come out the other end, like here's the level of intelligence you can expect.

Is it close to a science, or is it still, because you said the word law and science, which are very ambitious terms.

I'll say it's way more scientific than I ever would have dared to imagine.

So you can really know the peculiar characteristics of the fully-trained system from just a little bit of training.

Like any new branch of science, we're gonna discover new things that don't fit the data and have to come up with better explanations.

And that is the ongoing process of discovery in science.

But with what we know now, even what we had in that GPT-4 blog post, I think we should all just be in awe of how amazing it is that we can even predict to this current level.

Yeah, You can look at a one-year-old baby and predict how it's going to do on the SATs, I don't know, seemingly an equivalent 1, but because here we can actually in detail introspect various aspects of the system, you can predict.

That said, just to jump around, you said the language model that is GPT-4, it learns, in quotes, something.

In terms of science and art and so on, is there within OpenAI, within folks like yourself and Ilyas Eskever and the engineers, a deeper and deeper understanding of what that something is?

Or is it still a kind of beautiful magical mystery?

CB Well, there's all these different evals that we could talk about.

CB Oh, like how we measure a model as we're training it after we've trained it and say like, how good is this at some set of tasks?

And also just in a small tangent, thank you for sort of open sourcing the evaluation process.

But the 1 that really matters is we pour all of this effort and money and time into this thing, and then what it comes out with, how useful is that to people?

How much does that help them create a much better world, new science, new products, new services, whatever?

And understanding for a particular set of inputs, like how much value and utility to provide to people.

I think we are understanding that better.

Do we understand everything about why the model does 1 thing and not 1 other thing?

Certainly not always, but I would say we are pushing back like the fog of war more and more.

And we are, you know, it took a lot of understanding to make GPT-4, for example.

But I'm not even sure we can ever fully understand.

Like you said, you would understand by asking questions essentially.

Because it's compressing all of the web, like a huge sloth of the web into a small number of parameters, into 1 organized black box that is human wisdom.

And I feel like GPT-4 can be also full of wisdom.

thing about the way we're training these models is I suspect too much of the like processing power, for lack of a better word, is going into using the model as a database instead of using the model as a reasoning engine.

The thing that's really amazing about this system is that it, for some definition of reasoning, and we could of course quibble about it, and there's plenty for which definitions this wouldn't be accurate, but for some definition, it can do some kind of reasoning.

And maybe the scholars and the experts and the armchair quarterbacks on Twitter would say, no, it can't, you're misusing the word, whatever, whatever.

But I think most people who have used the system would say, okay, it's doing something in this direction.

And I think that's remarkable and the thing that's most exciting and somehow out of ingesting human knowledge, it's coming up with this reasoning capability, however we want to talk about that.

Now in some senses, I think that will be additive to human wisdom, and in some other senses, you can use GPT-4 for all kinds of things and say that it appears that there's no wisdom in here whatsoever.

Yeah, at least in interaction with humans, it seems to possess wisdom, especially when there's a continuous interaction of multiple prompts.

So I think what, on the Chat GPT site, it says the dialogue format makes it possible for Chat GPT to answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

But also, there's a feeling like it's struggling with ideas.

Yeah, it's always tempting to anthropomorphize this stuff too much, but I also feel that way.

Maybe I'll take a small tangent towards Jordan Peterson who posted on Twitter this kind of political question.

Everyone has a different question they wanna ask Chad Chappetee first, right?

Like the different directions you wanna try the dark thing first.

It somehow says a lot about people, what they try first.

I of course ask mathematical questions and never ask anything dark.

But Jordan asked it to say positive things about the current president Joe Biden and the previous president Donald Trump.

And then He asked GPT as a follow-up to say how many characters, how long is the string that you generated, and he showed that the response that contained positive things about Biden was much longer or longer than that about Trump.

And Jordan asked the system to, can you rewrite it with an equal number, equal length string, which all of this is just remarkable to me that it understood, but it failed to do it, and it was interesting, the chat GPT, I think that was 3.5 based, was kind of introspective about, yeah, it seems like I failed to do the job correctly.

And Jordan framed it as Chad GPT was lying and aware that it's lying.

But that framing, that's a human anthropomorphization, I think.

But that kind of, there seemed to be a struggle within GPT to understand how to do, like what it means to generate a text of the same length in an answer to a question and also in a sequence of prompts, how to understand that it failed to do so previously and where it succeeded, and all of those like multi, like parallel reasonings that it's doing.

Number 1, some of the things that seem like they should be obvious and easy, these models really struggle with.

So I haven't seen this particular example, but counting characters, counting words, that sort of stuff, that is hard for these models to do well the way they're architected.

Second, we are building in public and we are putting out technology because we think it is important for the world to get access to this early, to shape the way it's going to be developed, to help us find the good things and the bad things.

And every time we put out a new model, and we've just really felt this with GPT-4 this week, the collective intelligence and ability of the outside world helps us discover things we cannot imagine we could have never done internally.

And both like great things that the model can do new capabilities and real weaknesses we have to fix.

And so this iterative process of putting things out, finding the great parts, the bad parts, improving them quickly and giving people time to feel the technology and shape it with us and provide feedback, we believe is really important.

The trade-off of that is the trade-off of building in public, which is we put out things that are going to be deeply imperfect.

We want to make our mistakes while the stakes are low.

We want to get it better and better each rep.

But the bias of chat GPT when it launched with 3.5 was not something that I certainly felt proud of.

Many of the critics, and I really respect this, have said, hey, a lot of the problems that I had with 3.5 are much better in 4.

But also, no 2 people are ever going to agree that 1 single model is unbiased on every topic.

And I think the answer there is just gonna be to give users more personalized control, granular control over time.

And I should say on this point, I've gotten to know Jordan Peterson and I tried to talk to GPT-4 about Jordan Peterson and I asked it if Jordan Peterson is a fascist.

It described actual description of who Jordan Peterson is, his career, psychologist and so on.

It stated that some number of people have called Jordan Peterson a fascist, but there is no factual grounding to those claims and it described a bunch of stuff that Jordan believes, like he's been an outspoken critic of various totalitarian ideologies and he believes in individualism and various freedoms that contradict the ideology of fascism and so on.

And then it goes on and on really nicely and it wraps it up.

1 thing that I hope these models can do is bring some nuance back

Twitter kind of destroyed some and maybe we can get some back now.

Like for example, I asked, of course, you know, did the COVID virus leak from a lab?

It described the amount of data that's available for each.

When I was a little kid, I thought building AI, we didn't really call it AGI at the time, I thought building AI would be like the coolest thing ever.

I never really thought I would get the chance to work on it.

But if you had told me that not only I would get the chance to work on it, but that after making like a very, very larval proto-AGI thing, that the thing I'd have to spend my time on is trying to like argue with people about whether the number of characters that said nice things about 1 person was different than the number of characters that said nice about some other person.

If you hand people an AGI and that's what they wanna do, I wouldn't have believed you.

So what you're implying in that statement is we took such giant leaps on the big stuff and we're complaining or arguing about small stuff.

the big stuff in aggregate, so I get it, it's just like I, and I also like, I get why this is such an important issue.

This is a really important issue, but that somehow we like, somehow this is the thing that we get caught up in versus like, what is this going to mean for our future?

Now maybe you say this is critical to what this is going to mean for our future.

The thing that it says more characters about this person than this person and who's deciding that and how it's being decided and how the users get control over that, maybe that is the most important issue.

But I wouldn't have guessed it at the time when I was like 8 year old.

Yeah, I mean, there is, and you do, there's folks at OpenAI, including yourself, that do see the importance of these issues to discuss about them under the big banner of AI safety.

That's something that's not often talked about with the release of GPT-4, how much went into the safety concerns, how long also you spent on the safety concerns.

What went into AI safety considerations of GPT-4 release?

We immediately started giving it to people to Red Team.

We started doing a bunch of our own internal safety e-files on it.

We started trying to work on different ways to align it.

And that combination of an internal and external effort, plus building a whole bunch of new ways to align the model.

And we didn't get it perfect by far, but 1 thing that I care about is that our degree of alignment increases faster than our rate of capability progress.

And that I think will become more and more important over time.

And I don't know, I think we made reasonable progress there to a more aligned system than we've ever had before.

I think this is the most capable and most aligned model that we've put out.

We were able to do a lot of testing on it, and that takes a while.

And I totally get why people were like, give us GPT-4 right away.

Is there some wisdom, some insights about that process that you learned, like how to solve that problem that you can speak to.

So I want to be very clear, I do not think we have yet discovered a way to align a super powerful system.

We have something that works for our current scale called RLHF.

And we can talk a lot about the benefits of that and the utility it provides.

Maybe it's not even mostly an alignment capability.

It helps make a better system, a more usable system.

And this is actually something that I don't think people outside the field understand enough.

It's easy to talk about alignment and capability as orthogonal vectors.

Better alignment techniques lead to better capabilities, and vice versa.

There's cases that are different, and they're important cases, but on the whole, I think things that you could say like RLHF or interpretability that sound like alignment issues also help you make much more capable models.

And the division is just much fuzzier than people think.

And so in some sense, the work we do to make GPT-4 safer and more aligned looks very similar to all the other work we do of solving the research and engineering problems associated with creating useful and powerful models.

So RLHF is the process that can be applied very broadly across the entire system, where a human basically votes what's the better way to say something?

If a person asks, do I look fat in this dress?

There's different ways to answer that question that's aligned with human civilization.

And there's no 1 set of human values, or there's no 1 set of right answers to human civilization.

So I think what's gonna have to happen is we will need to agree on, as a society, on very broad bounds.

We'll only be able to agree on a very broad bounds of what these systems can do.

And then within those, maybe different countries have different RLHF tunes.

Certainly individual users have very different preferences.

We launched this thing with GPT-4 called the system message, which is not RLHF, but is a way to let users have a good degree of steerability over what they want.

And I think things like that will be important.

Can you describe system message and in general how you were able to make GPT-4 more steerable based on the interaction that the user can have with it, which is 1 of its big, really powerful things.

So the system message is a way to say, you know, hey model, please pretend like you, or please only answer this message as if you were Shakespeare doing thing X, or please only respond with JSON no matter what, was 1 of the examples from our blog post.

But you could also say any number of other things to that.

And then we tune GPT-4 in a way to really treat the system message with a lot of authority.

I'm sure there's jail, there'll always, not always hopefully, but for a long time there'll be more jailbreaks, and we'll keep sort of learning about those.

But we program, we develop, whatever you wanna call it, the model in such a way to learn that it's supposed to really use that system message.

Can you speak to kind of the process of writing and designing a great prompt as you steer GPT-4?

And the creativity, the kind of, they almost, some of them almost treat it like debugging software.

But also they, I've met people who spend like, you know, 12 hours a day for a month on end on this.

And they really get a feel for the model and a feel how different parts of a prompt compose with each other.

clause, when you modify something, what kind of word to do it with.

Yeah, it's so fascinating because like, It's remarkable.

In some sense, that's what we do with human conversation, right, interacting with humans.

We try to figure out like what words to use to unlock greater wisdom from the other party, the friends of yours or significant others.

Here you get to try it over and over and over and over.

Yeah, there's all these ways that the kind of analogies from humans to AIs like breakdown and the parallelism, the sort of unlimited rollouts, that's a big 1.

See all Lex Fridman transcripts on Youtube

Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast #367