Matt Wood is the VP of product at Amazon Web Services. He joins Big Technology Podcast live at the AWS Summit in New York City for a conversation about Amazon's plan to compete — and thrive — in the AI race. In this conversation, Wood describes Amazon's plan to support all AI models, not just one, and support builders via its computing infrastructure. Tune in for the most extensive comments from Amazon on its AI strategy to date. Stay tuned for the second half were we discuss ethics, company culture, and Jeff Bezos. 
You can read Alex's Substack story about the conversation here: Inside Amazon’s Low Key Plan To Dominate AI
---
Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.
For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/
Questions? Feedback? Write to: bigtechnologypodcast@gmail.com

An Amazon VP working directly on the company's AI initiatives joins us live from Amazon's AWS Summit in New York City.

All that and more coming up right after this.

Hi, I'm Ryan Roslansky, the CEO of LinkedIn.

Discover how today's greatest leaders shape their unique career paths and the lessons they learned along the way.

Tune into new episodes weekly starting August 8th, wherever you enjoy your podcasts.

Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond.

And we have a special show for you today.

We are live here at Amazon's AWS Summit at the Javits Center in New York City.

He's the VP of product working on AI products inside AWS.

So just so folks know, we have a live audience in front of us.

I want to make sure that you all get on the recording.

So you're going to let the people at home hear you.

So Matt, I did my research about where Amazon fits in the generative AI space and I looked at like the chat GPT model and I looked at chips.

Okay, you're dabbling in that area, in those areas, but I don't know if you stand out there yet, but where you are really trying to compete is in this space where companies, the big companies, bring their models inside AWS, and then anybody that wants to build something with an LLM can build it through your products.

So talk to us a little bit about that initiative and why did Amazon pick that in particular?

Number 1, I would say we're doing a lot more than dabbling.

I think we've got a very meaningful focus and investment just across the company on generative AI.

Myself, I think the rest of the team, I think like a lot of other people here, probably the light bulb came on when we started playing with chat GPT.

When that first came out, we really got excited and inspired by the capability here.

And so what we're trying to do is maybe a little different from what some other folks are doing.

What we want to do is take this technology and make it as broadly available as possible.

There's a lot of these kind of magical, interesting technologies like cloud computing 20 years ago, like machine learning 10 years ago, and now artificial intelligence, that have traditionally been only available to the very, very largest technology companies, to the biggest governments and academic agencies.

And so my mission and our approach is that we want to make that as broadly distributed as possible.

We want every builder and everyone to have access to the same capabilities that were once very, very limited.

Okay, wait, let me challenge you on that right off the bat.

I mean, there's a huge open source movement in this area.

In fact, Facebook just released this Lama 2 model, open source, you can use it, customize it.

So where is that gap that you're seeing between the high barrier and what is available on the market.

So, Lama 2 is an excellent, very capable model.

But there is a long way to go from having the model weights, which are what comprises the neural network, to actually building out an artificial intelligence system.

And just having the model weights is super useful, but it's like having the source code to some software.

Yes, it will give you some capability, but you still need to be able to deploy that somewhere.

You still need to be able to understand it enough to be able to make changes to it.

You need tooling that understands how it works so you can actually take it as an engine and put it inside the car that you're building.

And so what we're trying to do at AWS is make it really, really easy to take Lama 2 and other models like it, whether they're open source or proprietary, and use it as an engine inside cars and boats and planes and all sorts of things.

So tell me a little bit about the process.

So Facebook comes to you, or Meta comes to you, and says, hey, Matt, we have this cool open source model.

We'd like to make it available to your clients through AWS.

And what we do then is we take the model and the weights and we put it inside a capability, our machine learning service that we have called SageMaker.

And SageMaker lets any builder build, train, and deploy using machine learning models.

And LLAMA2 is 1 of several dozen large language models that are available on SageMaker today.

So that's almost exactly how it happened, actually.

Right, so SageMaker is your managed service that allows people to shape models.

But you have a newer product that's called Bedrock, which allows people to build things like agents, for instance.

get to pick, it's very interesting, They get to pick the different model that they want.

So talk a little bit about how that works.

And I'd really love to hear, you know, about Amazon's progression from SageMaker to Bedrock.

And why that puts you in a better strategic position.

Sure, I think SageMaker, we launched 2017, I think, and it's been very successful, very happy with the business, customers love it.

Many of our customers have standardized on SageMaker for their machine learning workloads, But 1 of the super interesting things about generative AI is inherently, because you're not training the models yourself, you're taking models from Amazon and Meta and a whole host of other stability AI and just building on top of them.

It makes machine learning far more accessible.

And so, while SageMaker is great at building and training and deploying those models, we wanted to find a way which gave the maximum leverage of that accessibility.

So you could just, Instead of having to figure out how the model works and fine tune your data and all those sorts of things, just give a prompt, just tell the system what you want, choose the model that you want to run it against.

We have about half a dozen models, plus include from partners and some from Amazon.

There's no servers, there's no infrastructure to manage.

You don't have to worry about using data or labeling data or worrying about GPUs or capacity or any of those things.

Just give us a prompt, put in, choose the model, and we'll give you the output.

Like if you wanted to make a chatbot, for instance, like people think, all right, I want to make a bot.

But what you could do actually in Amazon's technology is go ahead and build an agent or a bot and then pick whether you want open AI or Claude, right?

And then our approach is to find areas that are really, really valuable, the customers, real problems the customers are trying to solve, and then add capabilities to Bedrock to make those problems smaller.

So, chatbots, like you may have played with ChatGPT, they're very capable.

They can understand what you're talking about, they have context, you can go back and forth, and they give the appearance of intelligence, but they actually are not very good today at completing complex tasks.

And so let's say you wanted to create a right retirement plan.

You could ask your chatbot, build me a retirement plan.

And it would go off and it would build a very kind of reasonable approach to retirement planning.

But it will give you a pretty good strategy, a pretty good starting point.

But it doesn't know about your personal finances.

It doesn't know about the state of the markets.

It doesn't know what products are available.

And so 1 of the things we're adding to Bedrock is the ability to be able to provide that information to the language model.

Using your own private data inside the applications that are already running the Amazon Cloud.

And to be able to extend the language models capability with that data in couple minutes so that the model can produce, not just a strategy, but it can actually help you complete a task, and that's something that hasn't been possible before.

Then, I go back to the release of Lambda 2, last, well, a couple of weeks ago.

And okay, I saw it was definitely on AWS, but Azure is the preferred partner there.

It's a strategy that makes sense in my mind, but you're also, your competitors are doing it as well.

I think the distinction is that We have a slightly different approach to some other providers.

We want to broadly democratize this technology.

We want to do that because we think that there's not going to be a single model to rule them all.

So, others are talking about, Well, our stated goal is that we want an artificially, generally intelligent system.

Our stated goal is that we want to just be very pragmatic, meet customers where they're at today, and then provide capabilities like agents, provide the option of different models, and then allow customers to deploy those capabilities in a way which is very low cost, high availability, and low latency.

And that operational performance is, I think, something that's gonna really set folks apart in the future.

Okay, but I have to get back to this Azure example.

So, I think what you're basically saying is, you're not going to have this field to yourself.

You realize that you're going to come up against competitors doing the same thing, or maybe I'm putting words in your mouth.

Well, I think it's safe to say this is going to be a very competitive space for a very, very long time.

And so when you match early capabilities with a huge opportunity, you're naturally going to get a ton of different competition and ideas and thoughts and we have our own and who knows if they'll turn out to be right or not, but our key point of differentiation is to be able to allow builders to be able to build these systems with their own data, privately and securely, and to be able to leverage the investments that they've already made in that data on AWS, using completely novel capabilities, in addition to existing models and novel models as well.

So, there's a lot of focus on models today, but those models are going to remain important.

But over time, there's going to be additional capabilities, like agents, like reinforcement learning, like the ability to be able to understand and vet the responses that come out of these things, our accuracy, that are going to be as important as the models over time.

They're so deeply invested in OpenAI and those GPT models.

Okay, put me in the seat of a customer who's evaluating these 2 solutions.

OpenAI, Microsoft got all the buzz in the beginning.

But Microsoft is totally sort of all in on this model.

You have your own models, we're gonna talk about them.

But Amazon seems to be less, it has less at stake in terms of making yours work.

So is there a point of differentiation there, if I'm a customer, like for instance, is it more neutral?

We have our own models, yes, and we think that they're going to be very capable.

But we also recognize that different models will have different sweet spots.

Some are going to be really good at managing data.

Some are going to be really good at translating languages.

Some are going to be really good at reasoning.

And we expect that most customers are going to want access to not a single model that tries to do it all, but a range of models that are good at different things.

And I think today we're the only place where you can take data.

And we have customers with exabytes of data.

You'd be surprised how many customers have exabytes of data on AWS.

And they can take that data that they've invested in, and they can use it with these models to create a net new asset for their organization that is valuable and unique and private.

So I have a note here, and it's just like, man, you guys did Alexa first.

So why didn't you end up leading this LLM conversation?

I mean, the fact that we thought we were gonna be talking to intelligent assistants everywhere is kind of an Amazon idea.

it was an Apple idea, but their execution was bad.

You guys understood that we'd want to be talking to computers.

And yet, we just mentioned that you have your own LLM, your own model, that's right there for people to access in Bedrock.

Trust me, when I said I was going to be speaking with you, most of the general public was like, what's their strategy, where's their model?

So, just talk a little bit about what happened there.

Well, I think, number 1, we're very happy with Alexa.

Alexa is available to billions of customers and requests across hundreds of millions of endpoints.

So, listeners with 1 of those devices in your

Yes, we apologize, we're triggering the woman, or the man, if you've got it set that way.

So I think we're incredibly proud of the progress we've made with Alexa.

If you look at the way that it has been used in the real world, the number of end points that it's available in, If you'd have told me 6, 7 years ago that you could put Alexa in a microwave, put Alexa in a car, everywhere.

The devices should stay outside of the microwave.

So yeah, I think we're incredibly proud of that.

I think our vision for Alexa that we've always been striving for has been to provide a truly personal assistant.

Yeah, we talk about this idea of like the inspiration of coming from Star Trek and talking to the computer and all those sorts of things.

That's great, but I think our actual long-term vision is a really personal assistant, not a all-seeing controlling system.

It's not that Alexa should have been this.

It's that you had visibility into what this could be.

So I think that that, I think we should take inspiration from that.

It is a fantastic, probably 1 of the most remarkable technology demonstrations that I've ever seen.

There are plugins inside ChatGPT right now that allow you to do.

There's Code Interpreter that does some of the things that your models are working at, which is like speaking to data that it's not trained on and saying, bring me some results.

Well, Code interpreter actually just executes code which is generated by the model.

It doesn't have access to new information, doesn't have access to your own private information.

And on that point, nobody is putting their private information into ChatGPT.

There are hundreds, maybe thousands of CIOs that are telling that whole organization not to use ChatGPT because you-

The concern is accidental data exfiltration.

That when you're using ChatGPT, the service that exists on the web that we've all played with.

Whatever you type into that is being used to train and improve the models, which makes sense.

As a research tool, that makes a ton of sense.

But if you're an organization and you start to want to reason and understand or develop your own IP, that IP is not differentiated against, and it goes into the model and it gets exfiltrated.

We've seen actual customers see their own IP come back to them from the model.

That is terrifying to enterprise customers where the IP is the crown jewels.

And so, it is, I don't mean it in a diminutive way.

It is an excellent research tool from an exceptional research company.

I'm not taking anything away from what they have achieved, and I think they will continue to do wonderful work.

However, it is not the way that most organizations, in my opinion, are going to actually build and develop their generative capabilities.

And that's your bet, is to help them build these models and do it in a, I imagine, privacy safe way.

I'm making the pitch here, but I'm just trying to navigate this conversation and figure out what you guys are doing.

If a company comes to you and says, we want to incorporate LLMs in our business, you help them shape that, and so they don't end up dropping everything in chat GPT and having that spit out to their competitors.

The models have a similar level of capabilities, you're not really losing anything there.

But What we provide is security, what we provide is privacy.

So none of the information that is used with our Bedrock service is used to improve the underlying models.

In fact, none of that information even leaves your network.

You can see exactly what happens to that data and where it goes.

Then when you want to improve those models and customize them, specialize them for your own use cases internally, you don't want that specialization to be available to your competitors.

So we make it really, really easy to specialize those models in a way which gives you leverage against your own data and allows you to do that in a way which is completely private.

Because let me just tell you, Amazon, sorry, Andreessen Horowitz, I just read, they think that cloud on top of AI, the opportunity is 10-20 percent of this gen of AI spent.

If you look at the broader opportunity, I think that this technology is as transformational as the very earliest internet and the web browsers that allowed us to access it.

And it was that early internet and those early web browsers that gave inspiration and motivation and growth to companies like Amazon and Netflix and Airbnb.

And I think that there is going to be a wave of similarly Amazon-sized companies that evolve out of the generative AI opportunity generally.

And so I think we're gonna see multiple Amazon-sized organizations develop and grow over the next, who knows, 20 years, 10 years,

And you're content with 10 to 20% of that next boom?

I don't know if that's true, It seems very low to me, but I wouldn't be at all surprised if just the AI part of our cloud computing business was larger than the rest of AWS combined in a couple years.

No consumer product coming out from you guys.

Nothing to announce today, that's for sure.

I mean, We don't really operate in the search space.

We don't have a deep investment in web search.

Right, I'm just, I think you can expect for sure to see new invention and innovation coming from Alexa and devices and our retail stores and our ads business.

There isn't a team that I've spoken to at Amazon in the past 6 to 10 months that isn't really focused on understanding this technology and where it can be applied to their own business inside the company.

Microsoft recently said that they have 11, 000 customers who are using their OpenAI software building service.

Do you think, how important is it for a company to establish a lead in this moment?

And can you sit here and tell me with a straight face that Amazon is ahead of Microsoft?

And I don't think anybody, without a smile on their face, could call a winner 3 steps into a marathon race.

Amazon had this amazing moment where it got ahead.

Now, it wasn't everybody going at the same time and you were very early in AWS, so you know this, but AWS kind of ran away with the cloud computing field or cloud services field until other companies started to figure it out and by establishing itself so early, built that market dominance.

But I'm curious if you think we're going to see something like that in this moment or it just doesn't apply.

I think that there is going to be multiple, very credible options for builders.

And I am strongly convicted that AWS, if it is not the leader after today's announcements, which I would say it is, argue that it would be, but even if you take that as red, I think it's gonna be hard to argue that by the end of the year, you won't see AWS.

It will be hard to argue that AWS isn't in the top 1 or 2 providers.

Let's talk, you might enjoy this segment a little more, let's talk a little bit about...

Okay, well actually I'm gonna go back to my page of difficult questions.

Let's talk a little bit about building, right?

We have, we're in front of a room of developers, at least I hope, or other really very serious big technology fans.

So talk practically about what people have already done with your technology.

I mean, you had an announcement today that's kind of interesting about agents.

So I'd like to hear a little bit more about like the practical level of, maybe you can go step by step of like what, and briefly, but like what people would build with the AWS services.

See all Big Technology Podcast transcripts on Applepodcasts

Amazon Reveals Its AI Master Plan — With Matt Wood