- Dr. Andrew Wu discusses opportunities in AI, comparing it to a general purpose technology like electricity.
- He focuses on two important tools in AI: supervised learning and generative AI.
- Supervised learning is good at labeling things and has various applications such as spam detection and self-driving cars.
- Generative AI, like chat GPT, uses supervised learning to predict the next word and has potential as both a consumer and developer tool.
- Large language models can significantly speed up the development of AI applications, reducing the timeline from months to weeks.
- Dr. Wu believes this will lead to a flood of custom AI applications in the future.

- The speaker demonstrates how to write a sentiment classifier using Python and OpenAI tools.
- They emphasize that it is now easier and faster for developers to build systems like this.
- The speaker discusses the value of AI technologies, with supervised learning being the most valuable currently.
- They predict that generative AI will become more valuable in the next three years.
- The speaker highlights the potential for startups and large companies to create and capture value in the AI field.
- They mention the importance of identifying and executing on concrete use cases for AI technologies.
- The speaker talks about the potential for creating long-term value with deep and hard applications using AI.
- They discuss the challenges of AI adoption outside of consumer software internet and the need for better tools to enable customization.
- The speaker mentions low code and no code tools that allow users to customize AI systems without extensive coding.
- They emphasize the importance of enabling non-tech industries to leverage AI for their specific use cases.

- The speaker discusses two important trends in AI: its expansion beyond the tech and consumer software sectors, and the emergence of low-code, no-code tools for AI deployment in various industries.
- The speaker started AI Fund, a venture studio that builds startups to pursue diverse AI opportunities, as a way to efficiently pursue different AI projects in various industries.
- The AI stack includes the hardware semiconductor layer, infrastructure layer, developer tool layer, and application layer, with the application layer being crucial for the success of the entire stack.
- The speaker highlights the opportunities in the application layer, where competition is relatively lighter compared to the infrastructure and developer tool layers.
- The speaker shares the process of building startups, which involves validating ideas, recruiting a CEO, developing prototypes, and securing funding.
- An example is given of Bering AI, a company that uses AI to make ships more fuel efficient, showcasing the success of the startup-building process.
- The speaker emphasizes the importance of partnering with subject matter experts in different industries to create successful AI applications.
- The speaker prefers engaging in concrete ideas rather than exploring multiple alternatives during the ideation stage, as it is more efficient and cost-effective.

- Concrete ideas can be validated or falsified efficiently and give a clear direction to execute.
- Many subject matter experts have good ideas that no one is working on, and finding these ideas is more efficient.
- AI projects should be based on ethical grounds and address problems of bias and fairness.
- The disruption of jobs is a big risk of AI, particularly for higher wage jobs.
- Artificial general intelligence is still decades away and there is overblown hype about AI creating extinction risk.
- AI technology develops gradually and provides time for oversight and management.
- AI can be a key part of the solution to real extinction risks like pandemics and climate change.
- AI as a general purpose technology creates new opportunities for everyone.

- Dr. Andrew Wu discusses opportunities in AI, comparing it to electricity as a general purpose technology.
- He focuses on supervised learning and generative AI as important tools in AI.
- Supervised learning has applications like spam detection and self-driving cars.
- Generative AI, like chat GPT, uses supervised learning to predict the next word and has potential as a consumer and developer tool.
- Large language models can speed up AI development, leading to a flood of custom AI applications in the future.
- Dr. Wu demonstrates how to build a sentiment classifier using Python and OpenAI tools.
- AI technologies, particularly supervised learning, have significant value and generative AI will become more valuable in the next three years.
- Startups and large companies can create and capture value in the AI field.
- Concrete use cases and customization are important for AI adoption.
- Low code and no code tools enable customization of AI systems without extensive coding.
- AI is expanding beyond tech and consumer software sectors, and low-code, no-code tools are emerging for AI deployment in various industries.
- The AI Fund, started by the speaker, builds startups for diverse AI opportunities.
- The application layer in the AI stack has lighter competition compared to other layers.
- Building startups involves validating ideas, recruiting a CEO, developing prototypes, and securing funding.
- Partnering with subject matter experts is important for successful AI applications.
- Concrete ideas are more efficient and cost-effective during the ideation stage.
- AI projects should address bias and fairness issues.
- The disruption of jobs is a risk of AI, particularly for higher wage jobs.
- Artificial general intelligence is still decades away and there is overblown hype about AI creating extinction risk.
- AI technology develops gradually, allowing for oversight and management.
- AI can be a solution to real extinction risks like pandemics and climate change.
- AI as a general purpose technology creates new opportunities for everyone.

This discussion took place on July 26, 2023, at Cemex Auditorium, Stanford University, and was hosted by the Stanford Graduate School of Business.

This talk covers:
Trends in AI Technologies and Tools
Supervised Learning
Generative AI
The Adoption of AI
Opportunities in AI
Process for Building Startups
AI Risks and Social Impact

Explore the essentials of machine learning and develop practical skills to implement AI applications in this beginner-friendly program: 
https://www.coursera.org/specializations/machine-learning-introduction

Andrew is the Managing General Partner of AI Fund, founder of Deep Learning AI

and Landing AI, chairman and co-founder of Coursera, and an adjunct professor of computer science here at Stanford.

Previously he had started and led the Google Brain Team, which he had helped Google adopt modern AI, and he was also director of the Stanford AI Lab.

About 8 million people, 1 in 1,000 persons on the planet have taken an AI class from him.

And through both his education and his AI work, he has changed numerous lives.

So what I want to do today is chat to you about some opportunities in AI.

So I've been saying AI is the new electricity.

1 of the difficult things to understand about AI is that it is a general purpose technology, meaning that it's not useful only for 1 thing, but it's useful for lots of different applications, kind of like electricity.

If I was to ask you, what is electricity good for?

You know, it's not any 1 thing, it's a lot of things.

So what I'd like to do is start off sharing with you how I view the technology landscape, and this will lead into, the set of opportunities.

So a lot of hype, lots of excitement about AI.

And I think a good way to think about AI is as a collection of tools.

So this includes a technique called supervised learning, which is very good at recognizing things or labeling things, and generative AI, which is relatively new, exciting development.

If you're familiar with AI, you may have heard of other tools, but I'm gonna talk less about these additional tools.

And I'll focus today on what I think are currently the 2 most important tools which are supervised learning and generative AI.

So supervised learning is very good at labeling things or very good at computing input to outputs or A to B mappings.

For example, given an email, we can use supervised learning to label it as spam or not spam.

The most lucrative application of this that I've ever worked on is probably online advertising where given an ad, we can label if a user is likely to click on it and therefore show more relevant ads.

For self-driving cars, given the sense of readings of a car, we can label it with where are the other cars.

1 project that my team at AFM worked on was ship route optimization, where given a route that the ship is taking or considering taking, we can label that with how much fuel we think those will consume and use this, make ships, more fuel efficient.

Did a lot of work in automated visual inspection in factories.

So you can take a picture of a smartphone that was just manufactured and label is there a scratch or any other defect in it.

Or if you want to build a restaurant review reputation monitoring system, you can have a little piece of software that looks at online restaurant reviews and labels that as positive or negative sentiment.

So 1 nice thing, 1 cool thing about supervised learning is that it's not useful for 1 thing, it's useful for all of these different applications and many more besides.

Let me just walk through, you know, concretely the workflow of 1 example of a supervised learning, labeling things kind of project.

If you want to build a system to label restaurant reviews, you then collect a few data points, so collect the dataset where it say, you know, the pastrami sandwich great, you say that is positive, servants are slow, that's negative, My favorite chicken curry, that's positive.

And, here I've shown 3 data points, but you're building this, you may get thousands of data points like this, or thousands of training examples we call it.

And the workflow of a machine learning project, of an AI project is you get labeled data, maybe thousands of, you know, data points.

Then you have an AI engineering team train an AI model to learn from this data.

And then finally, you would find maybe a cloud service to run the trained AI model.

And then you can feed it, you know, best policy I've ever had, and that's positive sentiment.

And so, I think the last decade was maybe the decade of large scale supervised learning.

What we found starting about 10, 15 years ago, was if you were to train a small AI model, so train a small neural network or small deep learning algorithm, basically a small AI model maybe not on a very powerful computer, then as you fed it more data, its performance would get better for a little bit, but then it would flatten out, it would plateau and it would stop being able to use the data to get better and better.

But if you were to train a very large AI model, lots of compute on maybe powerful, you know, GPUs, then as we scaled up the amount of data we gave the machine learning model, its performance would kind of keep on getting better and better.

So this is why when I started and led the Google Brain team, the primary mission that I directed the team to solve at the time was, let's just build really, really large neural networks that we then fed a lot of data to and that recipe fortunately worked.

And I think the idea of driving large compute and large scale of data that recipes really helped us, driven a lot of AI progress over the last decade.

So if that was the last decade of AI, I think this decade is turning out to be also doing everything we had in supervised learning but adding to it the exciting too of generative AI.

So many of you, maybe all of you, will have played with chat GPT and bot and so on.

But Just, you know, given a piece of text which you call a prompt, like I love eating.

If you run this multiple times, maybe you get bagels, cream cheese, or my mother's meatloaf, or, out to friends and the AI system can generate output like that.

Given the amounts of buzz and excitement about Genes of AI, I thought I'd take just half a slide to, you know, say a little bit about how this works.

So it turns out that, Genes of AI, at least this type of text generation, the core of it is using supervised learning that inputs output mappings to repeatedly predict the next word.

And so if your system reads on the Internet a sentence like, my favorite food is a bagel with cream cheese and lox, then, this is translated into a few data points where If it sees my favorite food is a, in this case, try to guess that the right next word was bagel, or my favorite food is a bagel, try to guess next word is with, and similarly if it sees that, you know, in this case, the right guess for the next word would have been dream.

So by taking texts that you find on the Internet or other sources, and by using this input-output supervised learning to try to repeatedly predict the next word, if you train a very large AI system, on hundreds of billions of words.

So in the case of the largest models now more than a trillion words, then you get a large language model like ChagGP.

And, you know, there are additional other important technical details.

Technically, these systems predict the next sub-word, a part of word called the token.

And then there are other, techniques like, RHF for further tuning the AI output to be more, hopeful, honest, and harmless.

But at the heart of it is this using supervised learning to repeatedly predict the next word that, that's really what's enabling, the exciting, you know, really fantastic progress on large language models.

So, while many people have seen large language models as a fantastic consumer tool.

You can go to a website like Chack G.B.'s website or BODS or other large language models and use it as a fantastic tool.

There's 1 other trend I think is still underappreciated, which is the power of, large language models not as a cons- not as a- not just as a consumer tool, but as a developer tool.

So it turns out that there are applications that used to take me months to build, that a lot of people can now build much faster by using a large language model.

So specifically, the workflow for supervised learning, building the restaurant review system say, would be that you need to get a bunch of label data and, you know, maybe that takes a month to get a few thousand data points.

And then have an AI team train and tune and really get, you know, optimized performance on your AI model.

Then find a cloud service to run it, make sure it's running robustly, make sure it's recognized, maybe that'll take another 3 months.

So pretty realistic timeline for building a commercial grade machine learning system is like 6 to 12 months, right?

So teams I've led will often took, you know, roughly 6 to 12 months to build and deploy these systems and, and they- some of them turned out to be really valuable, but this is a realistic timeline for building and deploying a commercial grade AI system.

In contrast with prompt-based AI where you write a prompt, this is what the workflow looks like.

You can specify a prompt that takes maybe minutes or hours, and then you can deploy it to the Cloud and that takes maybe hours or days.

So there are now certain AI applications that used to take me, you know, literally 6 months, maybe a year to build, that many teams around the world can now build in maybe a week.

And I think this is already starting, but the best is still yet to come.

This is starting to open up a flood of a lot more AI applications that can be built by a lot of people.

So I think many people still underestimate the magnitude of the flood of custom AI applications I think is gonna come down the pipe.

Now, I know you probably were not expecting me to write code in this presentation, but that's what I'm going to do.

So it turns out this is all the code I need, in order to write a sentiment classifier.

So I'm gonna, you know, some of you will know Python I guess, import some tools from OpenAI, And then I have this prompt that says, classify the text below delimited by 3 dashes as having either a positive or negative sentiment.

Not only, I should, I don't know, learned a lot and also made great new friends.

And I've never run it before, so I really hope, thank goodness, we got the right answer.

And this is literally all the code it takes to build a sentiment classifier.

And so today, you know, developers around the world can take literally maybe like 10 minutes to build a system like this.

So, 1 of the things I've been working on was trying to teach online classes about how to use prompting, not just as a consumer tool, but as a developer tool.

So, to start off the technology landscape, let me now share my thoughts on what are some of the AI opportunities I see.

This shows what I think is the value of different AI technologies today, and I'll talk about 3 years from now.

But the vast majority of financial value from AI today is, I think, supervised learning, Where for a single company like Google can be worth more than $100 billion a year.

And also, there are millions of developers building supervised learning applications, so it's already massively valuable.

And also with tremendous momentum behind it just because of the sheer effort in, you know, finding applications and building applications.

And in generative AI is the really exciting new entrant, which is much smaller right now.

And then there are the other tools I'm including for completeness.

We can, you know, if the sizes of these circles represent the value today, this is what I think I might grow to in 3 years.

So supervised learning already really massive may double, say in the next 3 years from truly massive to even more massive.

And Genes of AI, which is much smaller today, I think will much more than double in the next 3 years because of the number of, amounts of developer interest, the amount of, venture capital investments, the number of large corporates exploring applications.

And I also just want to point out, 3 years is a very short time horizon.

If it continues to compound at anything near this rate, then in 6 years, you know, it'll be, it'll be even vastly larger.

But, this light shaded region in green or orange, that light shaded region is where the opportunities for either new startups, or for large companies incumbents to create and to enjoy value capture.

But 1 thing I hope you take away from this slide is that all of these technologies are general purpose technologies.

So in the case of supervised learning, a lot of the work that had to be done over the last decade, but is continuing for the next decade, is to identify and to execute on the concrete use cases, and that process is also kicking off for, generative AI.

So for this part of the presentation, I hope you take away from it that general purpose technologies are useful for many different tasks.

A lot of value remains to be created using supervised learning.

And, and even though we're nowhere near finishing, figuring out the exciting use cases of, supervised learning, We have this other, you know, fantastic tool of Genesive AI which further expands the set of things we can now do using AI.

But 1 caveat which is that there will be short-term fads along the way.

So I don't know if, some of you might remember the app called Lenza.

This is the app that would let you upload pictures of yourself and they'll render a cool picture of you as an astronaut or a scientist or something.

And it was a good idea and people liked it.

And it's roughly just took off like crazy like that through last December.

And that's because Lenzer was, it was a good idea.

But it was a relatively thin software layer on top of someone else's really powerful APIs.

And so even though it was a useful product, it was in a defensible business.

And when I think about apps like Lensr, I'm actually reminded of when Steve Jobs gave us the iPhone.

Shortly after, someone wrote an app that I paid $199 for, to do this, to turn on the LED, to turn the phone into a flashlight.

And that was also a good idea, to write an app to turn on the LED light.

But it also wasn't a defensible long term.

It also didn't create very long term value because it was easily replicated and underpriced and then eventually incorporated into iOS.

But with the rise of iOS, with the rise of iPhone, someone also figured out how to build things like Uber and Airbnb and Tinder.

The very long term, very defensible businesses that created, you know, sustaining value.

And I think with the rise of genitive AI or the rise of new AI tools, I think what really, what excites me is the opportunity to create those really deep, really hard applications that hopefully can create very long term value.

So the first trend I wanna share is AI as a general purpose technology and a lot of work that lies ahead of us is to find the very diverse use cases and to build them.

There's a second trend I want to share with you, which relates to why AI isn't more widely adopted yet.

It feels like a bunch of us have been talking about AI for like 15 years or something.

But if you look at where the value of AI is today, a lot of it is still very concentrated in consumer software Internet, right?

Once you go outside, you know, tech or consumer software Internet, there's some AI adoption but it, but a lot of use very early.

It turns out if you were to take all current and potential AI projects and sort them in decreasing order of value, then to the left of this curve, the head of this curve, are the multi-billion dollar projects like advertising or web search or for e-commerce, your product recommendations or company like Amazon.

And it turns out that about 10, 15 years ago, you know, various of my friends and I, we figured out a recipe for how to hire, say, 100 engineers to write 1 piece of software to serve more relevant ads and apply that 1 piece of software to a billion users and generate massive financial value, so that works.

But once you go outside consumer software internet, hardly anyone has 100 million or a billion users that you can write and apply 1 piece of software to.

So once you go to other industries, as we go from the head of this curve on the left over to the long tail, these are some of the projects I see and I'm excited about.

I was working with a pizza maker that was taking pictures of the pizza they were making because they needed to do things like make sure that the cheese is spread evenly.

But that recipe of hiring 100 engineers or dozens of engineers to work on a $5 million project, that doesn't make sense.

Or as another example, working with an agriculture company that with them, we figured out that if we use cameras to find out how tall is the wheat, and wheat is often bent over because of wind or rain or something.

And if we can chop off the wheat at the right height, then that results in more food for the farmer to sell and is also better for the environment.

But this is another $5 million project that that old recipe of hiring a large group of highly skilled engineers to work on this 1 project, that doesn't make sense.

And similarly materials grading, cloth grading, sheet metal grading, many projects like this.

So whereas to the left in the head of this curve, there's a small number of let's say multi-billion dollar projects, and we know how to execute those, you know, delivering value.

In other industries, I'm seeing a very long tail of tens of thousands of, let's call them $5 million projects that until now have been very difficult to execute on because of the high cost of customization.

The trend that I think is exciting is that the AI community has been building better tools that lets us aggregate these use cases and make it easy for the end user to do the customization.

So specifically, I'm seeing a lot of exciting low code and no code tools that enable the user to customize the AI system.

What this means is instead of me needing to worry that much about pictures of Pisa.

We have tools, because we're starting to see tools that can enable the IT departments of the Pisa making factory to train an AI system on their own pictures of Pisa to realize this $5 million worth of value.

And by the way, the pictures of Pizzer, you know, they don't exist on the Internet.

So Google and Bing don't have access to these pictures.

We need tools that can be used by really the Pizzer factory themselves to build and deploy and maintain their own custom AI system that works on their own pictures of Pisa.

And broadly, the technology for, for enabling this, some of it is prompting, text prompting, visual prompting, but really large language models and, and, and similar tools like that, or a technology called data-centric AI, whereby instead of asking the pizza factory to write a lot of code, you know, which is, which is, which is challenging, we can ask them to provide data which turns out to be more feasible.

And I think the second trend is important because I think this is a key part of the recipe for taking the value of AI, which so far still feels very concentrated in the tech world and the consumer software internet world.

And pushing this out to all industries, really to the rest of the economy which, which, you know, sometimes it's easy to forget.

The rest of the economy is much bigger than the tech world.

So, the 2 trends I shared, AI is a general purpose technology, lots of concrete use cases to be realized, as well as low-code, no-code, easily used tools enabling AI to be deployed in more industries.

How, how do we go after these opportunities?

So about 5 years ago, there was a puzzle I want to solve, which is I felt that many valuable AI projects are now possible.

And having led AI teams in Google and Baidu in big tech companies, I had a hard time figuring out how I could operate a team in a big tech company to go off there, a very diverse set of opportunities.

And everything from maritime shipping to education to financial services, to health care, and on and on.

It's just very diverse use cases, very diverse go to markets, and very diverse really, you know, customer bases and applications.

And I felt that the most efficient way to do this would be if we can start a lot of different companies to pursue these very diverse opportunities.

So that's why I ended up starting AI Fund, which is a venture studio that builds startups to pursue a diverse set of AI opportunities.

And of course, in addition to lots of startups, incumbent companies also have a lot of opportunities to integrate AI into existing businesses.

In fact, 1 pattern I'm seeing for incumbent businesses is distribution is, is often 1 of the cyclical advantages of incumbent companies is they play their cards right.

Can allow them to integrate AI into, into their products quite efficiently.

But Just to be concrete, where are the opportunities?

So I think of this as a, this is what I think of as the AI stack.

All right, at the bottom level is the hardware semiconductor layer.

Fantastic opportunities there, but very capital intensive, very concentrated.

So there's a lot of resources, relatively few winners.

So some people can and should play there.

I personally don't like to play there myself.

Also fantastic opportunities, but very capital intensive, very concentrated.

So I tend not to play that myself either.

And then there's the developer tool layer.

What I showed you just now was, I was actually using OpenAI's API as a developer tool.

And then I think the developer tool sector is hyper-competitive.

Look at all the startups chasing OpenAI right now, but there will be some mega winners.

And so I sometimes play here, but primarily when I think of a meaningful technology advantage, because I think that earns you the right or earns you a better shot at being 1 of the mega winners.

And then lastly, even though a lot of the media attention and the buzz is in the infrastructure and developer tooling layer, it turns out that that layer can be successful only if the application layer is even more successful.

And we saw this with the rise of SaaS as well.

A lot of the buzz and excitement is on the technology, the tooling layer, which is fine, nothing wrong with that.

But the only way for that to be successful is that the application layer is even more successful so that frankly, they can generate enough revenue to pay the infrastructure and the tooling layer.

Armor.ai, I was actually just texting the CEO yesterday, but Armor.ai is a, a company that we built that uses AI for romantic relationship coaching, right?

And just to point out, I'm an AI guy, and I feel like I know nothing really about romance.

And if you don't believe me, you can ask my wife.

She will confirm that I know nothing about romance.

But we want to build this, we wanted to get it together with the former CEO of Tinder, Renata Naibog and with my team's expertise in AI, and her expertise in relationships.

She knows more about, you know, relationships than I think anyone I know.

We're able to build something pretty unique using AI for, you know, kind of romantic relationship mentoring.

And the interesting thing about applications like these is when we look around, you know, how many teams in the world are simultaneously expert in AI and in relationships?

And so at the application layer, I'm seeing a lot of exciting opportunities that seem to have a very large market, but where the competition set is very light relative to the magnitude of the opportunity.

It's not that there are no competitors, but it's just much less intense compared to the developer tool or the infrastructure layer, I'd say.

And so, because I've spent a lot of time iterating on a process of building startups.

What I'm going to do is just, you know, very transparently tell you the recipe we've developed for building startups.

And so after many years of iteration and improvement, this is how, you know, we now build startups.

My Teams always had access to a lot of different ideas, you know, internally generated, ideas from partners.

And I want to walk through this with 1 example of something we did, which is a company Bering AI, which uses AI to make ships more fuel efficient.

So this idea came to me when a few years ago, a large Japanese conglomerate called Mitsui, that is a major shareholder in sort of operates a major shipping lines.

They came to me and they said, hey Andrew, you should build a business to use AI to make ships more fuel efficient.

And the specific idea was, you know, think of it as a Google Maps for ships.

We can suggest a ship or tell a ship how to steer, so that you still get to your destination on time, but using, it turns out about 10% less fuel.

And so what we now do is we spend about a month validating the idea.

So double check, is this idea even technically feasible?

And then talk to prospective customers to make sure there's a market need.

So we spend up to about a month doing that.

And if it passes this stage, then we will go and recruit a CEO to work with us on the project.

When I was starting out, I used to spend a long time working on the project myself before bringing on the CEO.

But after iterating, we realized that bringing on the leader at the very beginning to work with us, it reduces a lot of the burden of having to transfer knowledge or having a CEO come in and reval, having to revalidate whether we discover it, so the process is, we've learned much more efficient, we just bring in the leader at the very start.

And so in the case of Bering AI, we found a fantastic CEO, Dylan Kyle, who's a repeat entrepreneur, 1 successful exhibit before.

And then we spent 3 months, 6 two-week sprints to work with them, to build a prototype, as well as do deep customer validation.

If it survives this stage, and we have about a two-thirds, 66% survival rate.

We then write a first check-in, which then gives the company resources to hire an executive team, build a key team, get MVP working, minimum viable product working, and get some real customers.

And then after that, hopefully then successfully raises additional external rounds of funding and can keep on growing and scaling.

See all Stanford Online transcripts on Youtube

Andrew Ng: Opportunities in AI - 2023