- AI is being used to write speeches for weddings and as voice bots for answering machines.
- AI tools are good at creative storytelling and generating human-like voices.
- They are also useful for semantic search and answering questions about arbitrary data.
- Fine-tuning AI models is necessary to adapt to new types of data and trends.
- AI tools provide around 85% to 90% solutions, but additional fine-tuning is required for genuine value.
- Giving AI structure and specific tasks improves reliability.
- The iterative process is important, as data and model quality can change.
- AI can introduce randomness into outputs, allowing for exploration and learning.
- However, AI can also hallucinate and struggle to distinguish fact from fiction.
- Trustworthiness and human supervision are still challenges in using AI.
- Human involvement is crucial for assessing corrections and avoiding hallucinations.
- The goal is to develop AI technology that deepens human connection and serves humans.

At a recent meetup for Y Combinator founders working in AI, we asked them about the current state of AI technology—the good, the challenges, and best practices. Their expertise should offer some insight for anyone interested in building a product in the artificial intelligence space.

Thank you to all the companies who volunteered to be interviewed for this episode.

Apply to Y Combinator: https://yc.link/FounderFAQ-apply 
Work at a Startup: https://yc.link/FounderFAQ-jobs

Chapters (Powered by https://bit.ly/chapterme-yc) - 
00:00 - Intro
00:58 - What are current AI tools good at?
04:24 - Hallucination
06:00 - Human-in-the-loop
06:53 - Outro

What is the most maybe unexpected way that you use AI in your regular life?

Yeah, I don't know if I should say this, but writing speeches for weddings.

Yeah, we both set our answering machines to our voice bot.

This was, I think, programmed to talk about before.

Now we have it open every day to help us code.

It's just making me code infinitely faster.

You can describe to the AI what change you want to make to your UI, like build dark mode, and it'll edit all the code to implement that feature.

These tools will make humans more and more like narrators, like people who are describing what they want.

Then the model will actually create something that's even better than what humans would do if they're doing it themselves.

And I think the fundamental problem-solving skills are always going to be like really needed.

And understanding the technology and its constraints and how to leverage it is going to be so important.

I think 1 of the things that's really counterintuitive about generative AI tools is that they're really good at what we thought they would be really bad at, which is sort of creative storytelling work.

The goal is to make software so that anyone can make South Park from their bedroom.

Example, you can make a photo of me, Eric, and Rihanna playing volleyball on the beach.

That one's my favorite, or I like this 1.

We are building truly human-like AI voices that's like very conversational like humans.

Now, it's just indistinguishable from a human voice.

For us personally, what it's amazing at is semantic search.

That's something that didn't really work before.

Just like taking a random piece of text and finding relevant things.

They're pretty good at being able to take arbitrary data and be able to answer questions about it.

Because there's so much new type of data, it's really hard to adapt.

And so we are currently fine-tuning all of our models on these new types of data to make it as accurate as possible.

For fashion, very specifically, there are new terms popping up all the time.

You have to keep updating these models to know, oh, for this month, the trend is mermaid court, but for next month, maybe it's ballet court.

AI tools are pretty good in giving you around like an 85% to 90% solution.

But there's a lot more fine tuning or a lot more hacks that you need to put in place on top of them to ensure that you can deliver genuine value.

You can use a bunch of simple operations to actually do something really complicated.

You need to really give them structure about how it should look like and give them 1 particular task to do and then they do it very well.

If you're able to think through the process that you go through, then you can actually engineer a prompt or engineer like a sequence of steps so that you can have that entire process be even more reliable than you would be.

It's important to just be very iterative in your process and just debug and iterate on your prompts as you go.

If you think you have a solution, it may not be the same solution over time.

Your data can change, the actual underlying model quality can change with that.

And so the biggest difference is just, there's this sort of iteration required.

I think the hardest part is you're trying to marry deterministic kind of software with probabilistic models and we sit right at the middle of that.

It is like quite an exciting thing to work with because in the past with programming, the computer really just followed your instructions to a T, and you can expect the same results given the same inputs.

Now you put in the same inputs, you might get some variance.

If we can actually introduce some randomness into our outputs, then we can explore our space a bit better, and our models will get better from learning from all of these other choices that we can make.

It's not reliable in the way that you expect it to be reliable, Which is great for us, because for us, we're doing entertainment.

So it's like, as long as it's funny, it doesn't matter.

But I guess if you're operating a car, that seems more complicated.

Sometimes it can hallucinate and make up something that wasn't intended.

I would define a hallucination as AI generating something that doesn't exist, but looks like it might or should exist.

They're still really bad at distinguishing fact from fiction, so they're good creative storytellers, but they're surprisingly bad at knowing the difference between what's true and false.

If you're a doctor, finding out what GPT decided was the diagnosis for this patient probably takes a ton of time to verify.

And if there's any mistake, then you're in a lot of trouble.

At what point Do you trust the AI over the doctor?

There's been a lot of effort in the industry recently to sort of prevent these hallucinations, but that's created this opposite problem, which is now it will often think things aren't real or pretend that it doesn't know things that it really should do, right?

It will tell you it's like never heard of that article, even though it's definitely in the training set, right?

A bit like a human, when you read things and you take something away from that and internalize it, you can't necessarily exactly remember where you read it.

And so when you're using these models with real-world data, it's actually even harder to disambiguate what's a hallucination versus something that was a nuance in some piece of data.

You can't ask it for citations consistently.

That's still a challenge, and so the trustworthiness there has some way to go.

to say, like, hey, the accuracy metrics are better.

You have to understand that there's more at play, especially for human trust.

And that is a key component if you're going to develop a technology that people are going to use at the end of the day.

There's still a lot of nuance where we have to like steer them and that's why you'll hear a lot about this human in the loop.

It's really important to still have a human in the loop.

Having humans in the loop to initially assess whether the corrections that were needed to be made were accurate or not.

There needs to be someone supervising the output, making sure that there are no hallucinations.

It's really about figuring out the right ways to steer it.

And that's the challenge that I think all the YC companies working on AI are facing.

We're given this new tool to work with and we're all really just trying to figure it out.

I never want to lose sight of the fact that ultimately this is technology and service of humans and we get to keep humans to say the so what.

Ideally it's actually like deepening human connection where it's a lot more about


Speaker 0: What is the most maybe unexpected way that you use AI in your regular life?

Speaker 1: Yeah, I don't know if I should say this, but writing speeches for weddings.

Speaker 2: Yeah, we both set our answering machines to our voice bot. Hey, how's it going?

Speaker 3: The Hort is strong, Padawan.

Speaker 2: Oh, nice. This was, I think, programmed to talk about before.

Speaker 4: Yes, it was.

Speaker 5: Now we have it open every day to help us code.

Speaker 6: Helps us code.

Speaker 0: It's great

Speaker 4: at coding.

Speaker 1: It's just making me code infinitely faster. You can describe to the AI what change you want to make to your UI, like build dark mode, and it'll edit all the code to implement that feature.

Speaker 7: These tools will make humans more and more like narrators, like people who are describing what they want. Then the model will actually create something that's even better than what humans would do if they're doing it themselves.

Speaker 8: And I think the fundamental problem-solving skills are always going to be like really needed. And understanding the technology and its constraints and how to leverage it is going to be so important.

Speaker 6: What are current AI tools good at?

Speaker 3: Yeah, really, really good question. I think 1 of the things that's really counterintuitive about generative AI tools is that they're really good at what we thought they would be really bad at, which is sort of creative storytelling work. AI for creativity.

Speaker 0: The goal is to make software so that anyone can make South Park from their bedroom.

Speaker 9: Example, you can make a photo of me, Eric, and Rihanna playing volleyball on the beach. I like this 1.

Speaker 10: That one's my favorite, or I like this 1.

Speaker 11: I like this 1.

Speaker 7: We are building truly human-like AI voices that's like very conversational like humans.

Speaker 4: Without AI, the voices sounded horrible. Now, it's just indistinguishable from a human voice.

Speaker 8: For us personally, what it's amazing at is semantic search. That's something that didn't really work before. Just like taking a random piece of text and finding relevant things.

Speaker 0: LLMs have great ability to read.

Speaker 8: They're pretty good at being able to take arbitrary data and be able to answer questions about it.

Speaker 10: Because there's so much new type of data, it's really hard to adapt. And so we are currently fine-tuning all of our models on these new types of data to make it as accurate as possible.

Speaker 12: For fashion, very specifically, there are new terms popping up all the time. You have to keep updating these models to know, oh, for this month, the trend is mermaid court, but for next month, maybe it's ballet court.

Speaker 13: AI tools are pretty good in giving you around like an 85% to 90% solution. But there's a lot more fine tuning or a lot more hacks that you need to put in place on top of them to ensure that you can deliver genuine value.

Speaker 14: You can use a bunch of simple operations to actually do something really complicated.

Speaker 4: You need to really give them structure about how it should look like and give them 1 particular task to do and then they do it very well.

Speaker 14: If you're able to think through the process that you go through, then you can actually engineer a prompt or engineer like a sequence of steps so that you can have that entire process be even more reliable than you would be.

Speaker 1: It's important to just be very iterative in your process and just debug and iterate on your prompts as you go.

Speaker 6: If you think you have a solution, it may not be the same solution over time. Your data can change, the actual underlying model quality can change with that. And so the biggest difference is just, there's this sort of iteration required.

Speaker 15: I think the hardest part is you're trying to marry deterministic kind of software with probabilistic models and we sit right at the middle of that.

Speaker 0: It is like quite an exciting thing to work with because in the past with programming, the computer really just followed your instructions to a T, and you can expect the same results given the same inputs. Now you put in the same inputs, you might get some variance.

Speaker 5: If we can actually introduce some randomness into our outputs, then we can explore our space a bit better, and our models will get better from learning from all of these other choices that we can make.

Speaker 11: It's not reliable in the way that you expect it to be reliable, Which is great for us, because for us, we're doing entertainment. So it's like, as long as it's funny, it doesn't matter. But I guess if you're operating a car, that seems more complicated.

Speaker 0: So it is a double-edged sword. Sometimes it can hallucinate and make up something that wasn't intended.

Speaker 1: I would define a hallucination as AI generating something that doesn't exist, but looks like it might or should exist.

Speaker 3: They're still really bad at distinguishing fact from fiction, so they're good creative storytellers, but they're surprisingly bad at knowing the difference between what's true and false.

Speaker 2: If you're a doctor, finding out what GPT decided was the diagnosis for this patient probably takes a ton of time to verify. And if there's any mistake, then you're in a lot of trouble.

Speaker 16: At what point Do you trust the AI over the doctor?

Speaker 3: There's been a lot of effort in the industry recently to sort of prevent these hallucinations, but that's created this opposite problem, which is now it will often think things aren't real or pretend that it doesn't know things that it really should do, right? It will tell you it's like never heard of that article, even though it's definitely in the training set, right? Like, it's got to be there.

Speaker 17: A bit like a human, when you read things and you take something away from that and internalize it, you can't necessarily exactly remember where you read it. And so when you're using these models with real-world data, it's actually even harder to disambiguate what's a hallucination versus something that was a nuance in some piece of data. You can't ask it for citations consistently. That's still a challenge, and so the trustworthiness there has some way to go. It's not enough just

Speaker 16: to say, like, hey, the accuracy metrics are better. You have to understand that there's more at play, especially for human trust. And that is a key component if you're going to develop a technology that people are going to use at the end of the day.

Speaker 0: There's still a lot of nuance where we have to like steer them and that's why you'll hear a lot about this human in the loop.

Speaker 5: It's really important to still have a human in the loop.

Speaker 13: Having humans in the loop to initially assess whether the corrections that were needed to be made were accurate or not.

Speaker 5: There needs to be someone supervising the output, making sure that there are no hallucinations.

Speaker 0: So there are a lot of pros and cons. It's really about figuring out the right ways to steer it. And that's the challenge that I think all the YC companies working on AI are facing. We're given this new tool to work with and we're all really just trying to figure it out.

Speaker 15: I never want to lose sight of the fact that ultimately this is technology and service of humans and we get to keep humans to say the so what.

Speaker 17: Ideally it's actually like deepening human connection where it's a lot more about

Speaker 16: you

See all Y Combinator transcripts on Youtube

40 AI Founders Discuss Current Artificial Intelligence Technology