- The speaker discusses the differences between conducting user surveys in person versus online
- They mention that online surveys can be effective, but in-person interviews allow for more powerful interactions
- They emphasize the importance of regularly interacting with users to prevent product decline
- The speaker advises on how to handle situations where the users of a product are different from the buyers
- They suggest conducting interviews with different people in the buying chain to gain context and understand motivations
- The speaker also recommends using a champion user to gather information about all involved parties and their motivations
- When it comes to segmenting users for interviews, the speaker suggests using tools like Google Consumer Surveys and SurveyMonkey Audience
- They also mention the value of directly reaching out to users and having personal conversations with them
- The speaker advises on the importance of tabulating customer characteristics and patterns to identify the best customers to interview
- They recommend conducting user interview sessions once a week with a handful of people
- The question of when to do the sales pitch during user interviews is addressed, with the suggestion of focusing on understanding the users first before introducing the product
- The speaker suggests finding more people who reflect the same demographic as the best users and utilizing online platforms they frequent.

- The speaker discusses the strategy of using user interviews to find out what services users already use and to build a product based on their feedback.
- They emphasize the importance of not immediately pitching their product to users, but rather waiting a day or two and then presenting it as a result of the conversation.
- The speaker argues against labeling a product as beta, as it creates the perception that the product is not ready and can limit growth. They believe that all products have bugs and it is better to listen to user feedback and fix issues as they arise.
- They advise starting to build a product after gaining insights from user interviews, but caution against waiting for a specific number of interviews.
- The speaker suggests charging customers as early as possible, even before the product is fully developed, to gauge interest and commitment.
- They discuss the idea of using open source libraries as a way to build an audience and eventually monetize the product. They mention examples such as GitLab and Segment.

- Segment started as an open source project called analytics.js that duplicated website analytics to different aggregators
- The project gained popularity on Hacker News and became the foundation of the company
- It is important to start talking to customers from day 1, even if the MVP is not fully developed
- Validating customer interest can be done by creating a hack together solution using existing technology
- Dropbox launched with an After Effects movie demo and generated excitement without writing much code
- Instead of relying on expensive ads on platforms like Instagram and Facebook, it is more cost-effective to talk to users directly and infiltrate relevant communities
- Scalable methods for reaching interested users include physical methods like hanging door tags and digital methods like joining relevant Facebook groups
- The lecture concludes with a Q&A session and encourages attendees to network with each other.

The speaker discusses the differences between conducting user surveys in person versus online. They emphasize the importance of regularly interacting with users to prevent product decline. The speaker advises on how to handle situations where the users of a product are different from the buyers. They suggest conducting interviews with different people in the buying chain to gain context and understand motivations. The speaker also recommends using a champion user to gather information about all involved parties and their motivations. They recommend conducting user interview sessions once a week with a handful of people. The speaker suggests finding more people who reflect the same demographic as the best users and utilizing online platforms they frequent. They discuss the strategy of using user interviews to find out what services users already use and to build a product based on their feedback. The speaker argues against labeling a product as beta and advises starting to build a product after gaining insights from user interviews. They suggest charging customers as early as possible, even before the product is fully developed, to gauge interest and commitment. The lecture concludes with a Q&A session and encourages attendees to network with each other.

YC Partners Kevin Hale, Adora Cheung and Eric Migicovsky answer questions from the audience after our first week's lectures.

It's not too late to sign up! Register now through August 4th at https://www.startupschool.org/.

Have you experienced any difference between doing an in-person interview versus a online survey?

So the question was, have I experienced any differences between running user surveys in person or online.

Honestly, you'll probably figure out what the best solution is for yourself.

I think it's, again, it depends kind of on the context of the problem.

Sometimes I really find it powerful to stand behind a user and ask them to use their existing tools.

That can only really be done kind of well if you're in person, But I have seen some people do that pretty effectively over Zoom as well.

There's a UX researcher named Jared Spool and he talks about some research where he says like he noticed that in product teams, products tend to get worse over time if you find out that the product and design teams are not interacting directly with their users a minimum of every 6 weeks.

And so it's 1 of those things where it's like, there's just so much to be gained from just talking and having a small conversation, where you just miss a ton of stuff.

Because it's like, I don't know how many surveys you fill out, but the people who do tend to fill surveys, they fill a very weird bucket.

Yeah, no, you definitely need this open-ended conversation with a user rather than like a survey.

Do you have any examples of when your user is different from your buyer?

So for example, my startup involves potentially selling to countries, so I will be trying to get entire countries to sign arguments to buy from me, but they will be, the users of the product are going to be the people in the country, so who am I?

Yeah, the question was, what happens if the users of your product are different from the buyers?

And in the context of the asker, she was asking whether or countries may be the people using it but the the buyer may be an individual.

This usually calls for more user interviews.

So the best answer here is just to have conversations with different people in the buying chain.

Remember that a great user interview is not actually a sales pitch.

It's actually learning more about the context in which that person encounters the problem.

So in your case, I might consider having several different layers of interviews with potential buyers, potential users of the product, potential influencers, oftentimes in an enterprise sale, There are some people in the buying chain that aren't even involved in using the product whatsoever, but they have a say in maybe the budget or the time frame in which they'll purchase or use new solutions.

And so it helps to actually have conversations with them as well so you can get context around maybe past attempts that they've tried to solve this problem you know millions of dollars that they've already spent that went down the drain and you can learn like how they may be helpful or how they may hinder your sales process in the future.

Just a quick add to that I would say for the economic buyer in the company or whoever, government or whatever it is, talking to them and trying to understand how they're going to make the decision and what they actually value is probably most important.

The only thing I would add to that is that when you're talking to your user, there's usually in a complex B2B enterprise transaction, you have someone that's acting as your champion.

Someone who's super excited about using the product, usually the user, or there's someone who's like, I don't have to do this anymore if you get implemented or bought by my company.

And so you need to use them as your Virgil.

You're going to basically say, help me map out who's all involved, who needs to be in the room, and what are their motivations.

And so you're basically looking to get Intel for all that process.

And then the whole question is, you need to have all the arguments for things that they care about.

1 of the hacks that I like to use at this stage, and it helps if you're still in university, but you can actually talk to big companies as if you were, say the reporter for a school newspaper, or if you were working on a school project, you can use this as like a hack to actually get a pretty high level contact with someone who might become a champion, simply by kind of saying, oh, like I'm just really excited about budget management software.

And so you can actually interact and talk to people at a pretty high level before you even announce that you're working on a solution for that.

It's like how much money do you control for your budget?

So, for some of us, we've already launched and we're already talking to a bunch of users a week, but as we scale, that might get beyond what any 1 person can talk.

How do you isolate down the subset of users that you want to interview?

The question is, when we get to scale, how do you segment just the users?

What's the reason for segmenting the users?

Well, if you have a broad user base, if you talk to housewives in Nova Scotia, they might have different feedback than someone in New York.

So if you only talk to those housewives, you don't get feedback from the New Yorks.

How do you decide, all right, do you just get a little bit of everyone?

So there's a couple good tools for helping.

The question is basically, let's say you have a really broad audience that you're trying to figure out, what are their use cases, how do you best handle that segmentation so that way you get useful feedback to know specific direction of the product.

So there's a couple useful tools that I've used in the past and I've recommended to other people.

So Google Consumer Surveys and SurveyMonkey Audience.

So both of those you can ask really simple questions and choose the demographic.

SurveyMonkey is actually a little bit more expensive, but they actually have proper survey methodology that actually will balance it out based on like US population.

So for example, most people who fill out surveys, let's say for money or for non-profits, it's mostly skewed towards women, and they tend to be older, and so like if you were to ask Google consumer surveys, a lot of that stuff will tend to be skewed that way, and for them you can actually specifically handle it, and they'll actually use statistics to sort of balance it out.

So that's number 1, it's like directly asking.

I tend to not for a broad product do like a general surveys.

Usually it's almost always to my direct users to figure out, okay, who seems to be just already a fan of my product?

Who have I accidentally acquired who's a super fan?

I'm trying to figure out, how do I get more of you?

And then I'm doing everything I can to get them to talk to me.

So I'd rather, I will spend like 50 bucks on a gift card to get that user to have a conversation with me so I understand who he is and see if there's ways that I can multiply him.

the things to remember is never, Remember that as the founder of a company, you can actually reach out directly, personally, to any 1 of your users and ask if you could just jump on the phone and chat with them.

This is something that is kind of crazy because no big company, like Microsoft's not gonna do this, Facebook's not gonna do this, this is a competitive advantage that you as kind of a small early stage company have.

You as the product creator, the founder, can actually have a direct connection with any user in your user base.

Some people productize this and do like tons and tons of emails, but Others just choose to have maybe 10 conversations, 20 conversations with individual users every week.

Yeah, 1 of the things I always do, if you have enough data in your case, like lots of customers, is to start tabulating for each customer, figure out the characteristics of each 1, like female, where they live, income level, so and so forth.

And if you have all the emails, there's actually services out there that you can qualify all these people.

And then after that, bucket all the customers into like, here are my best customers, here are the ones that paid me 100 bucks or more, and then try to find patterns, and then go segment that way and talk to those people.

And like Kevin said, how do I get more people who naturally pay me more, what are the patterns, what do they do every day, and offer them gift cards or whatever to do user interview sessions.

I would aim, like if you have enough data and

people, I would really aim to do a session once a week with a handful of people.

I find that's the most informative thing and the best thing you can do for your business and move forward.

It doesn't have to be in person, depending on the product.

If you have an MVP and you're talking to users, trying to identify the problem, the whole point is what approach do you have to introduce this product that's built for them to start using it?

So let's say you have an MVP build, you're starting to do this user interview, you're listening and not talking.

To play off of Dora's point, once you begin to understand within your user base who are the best users, who are the people that are most disappointed if they couldn't use your product anymore, who are the ones that get the most benefit, you can actually go sideways.

And you can try to dig deep and find more people that reflect the same kind of demographic.

Some of the tips or tricks that we use is like, where do those people live on the Internet?

Which kind of other services do they already use?

And then you can like use the fingerprint of the users you already know to find more and find out where they live online.

So my strategy would be, if I was in that case and I had a wonderful user interview with them.

And it's looking like, oh God, what I've built is perfect for this person.

So instead of being like, I feel like it's the wrong moment to be like, ah, I gotcha, here's my thing.

I'd rather be like, I'd go home, maybe wait a day or 2 and be like, oh my god, thank you so much for the conversation that we had, and based on what you talked about, I built the first version of this.

I would love for you to check it out and let me know what you think.

And it's more likely that they're gonna be honored, their ego is stroked, and they're more likely to use it.

So to me, I feel like that gives you maximum benefit versus it feels like, did you even care about me at all?

Was this really a sit sales pitch hidden as a?

Exactly, and that's what you don't want to have after like being such a conscientious like empathetic listener

And no one's that quick a programmer that they programmed it while they were talking

I was at the AMA a few days ago, and I saw you make a comment regarding beta releases.

And I was curious to hear more of your thoughts about why you shouldn't release a beta as opposed to, you know, you want to limit the amount of people that are on your platform and just have a few people to talk to.

So in the AMA, I kind of poo-poo beta versions, or labeling your product as beta to artificially constrain growth on your product.

And so the main idea for me is that it creates like an affordance, it basically is a sign that says my product's not ready for you, don't use me because it's not ready for production, expect bugs, et cetera.

And the reason a lot of founders or engineers like to put beta on theirs because it's like, hey, you know what?

I built it myself, I know how many bugs there are and I'm so scared of people being disappointed.

So if I put this beta on, it's just going to remove all liability.

And to me, I feel like there's no product I've ever seen in production that doesn't have bugs.

And almost always what good founders do is they listen to the people, they respond to the bugs, they fix it as quick as possible.

And even in your beta, you will probably do the same.

So to me, I see no reason why you need to do that, is to put that beta on.

So I don't like anything that artificially can change growth, because all that stuff you're going to do for the beta to be a good founder, listening, fixing, et cetera, you're gonna do that even in production.

The second thing about beta is that you're gonna spend your whole company's life trying to get as many users as possible.

And so if you're thinking, I've built something so good that if I don't put beta on there, I'm just going to be overwhelmed with demand, I'm going to tell you that the hypothesis is probably wrong at all.

And so again, if something does go wrong or some issue that you're actually super scared of, like don't have that feature revealed.

If something does go wrong, just like, hey, thank you so much, I've only been working on it a week or I've only been working on it a month, I'm gonna fix it really quickly and then get it back to you.

Please let me know if you find anything else.

And then you're more likely to get a more positive response and people actually using it the way that you're hoping you might find people use it as a result.

I like real world honest tests about do people want this thing and then is my thing ready?

Yeah, the real fear that founders need to get over is, okay, I've built something, crap, people are gonna come.

If it's not good enough, they're never gonna come back.

Hopefully you're building a business that millions of people can use anyway.

And so just the 4 or 5 people that come back, or sorry, the 4 people that come and don't have a good experience, get that feedback and then just iterate towards something better.

You don't have to worry about them not coming back.

And I think that's actually 1 of the main reasons why most people don't launch early.

So how do I tell I can stop interviewing and then just start building?

But like, many people say it's 50 or 100, but what is a real moment that I can start building?

To me, they're gonna happen in concert with 1 another.

Oh, yeah, so the question is, like, okay, how many interviews do I have to do before I start building, et cetera?

And so if you don't have anything built and you don't even really know what the solution needs to be, You're just trying to figure out what is the problem set out there.

You basically do user interviews until you have an insight, be like, I think I can build something that will solve the problem, and then you immediately start building.

But for the most part, you want to do them in parallel.

Kevin offered a framework to think about, is the problem that I'm facing, or is the solution to the problem big enough to be a company?

You can actually extract that from those early user interviews more effectively than after you have a product because you're not biased by thinking like, I already like this, I have a great idea already.

You can actually be in that pure mode where it's like, is this a real problem?

Is this really a burning problem that people are facing?

And so if you can get away with taking a pre-order, wonderful.

We often give advice to companies who are considering making a complex B2B enterprise app, we're like, go try to sell that first before you go commit 6 to 12 months making

It's asking people to put their money where their mouth is.

Remember how I said before that oftentimes users don't really know what they want.

They won't be able to describe the future product, but they will be able to know it when they see it.

And at that point, you can extract data from them by saying, well, this person has the problem so acutely that they want to solve it, that they're willing to pay some random startup that doesn't even have a product yet.

That's a very strong signal that you're onto something.

Eric's model for talking to users is very similar to what the best salespeople actually do when they're trying to sell their product.

They're truly trying to understand the users or potential customers' problems, and then they're immediately trying to craft is like, oh yeah, we've got that.

They're able to reposition their product back to them.

And so this is why you're able to actually, for even complex enterprise software, if you truly understand the user or talk to them, you're like, oh, absolutely, we could totally build that.

We can have that at this time frame and I think we can do it at this cost.

Would you like to sign a contract and have us committed?

The whole thing is like, through the idea, you can sell it to the right customer if they believe that you're capable and that you truly are going to build something of value to them.

And so I did something where, like for my first product even before Pebble, we set up a landing page with Shopify, or we set up a sales cart where people could actually go through the checkout process to buy the product.

We didn't feel comfortable about charging people's credit cards, but we made the checkout process look exactly like they would have to pay.

So we figured we would get a better signal from those users, instead of just like an email sign up form, like, hey, are you interested?

They actually went through the flow thinking that they were going to have to pay something.

And at the end, we said, oh, we're actually accepting payment in a couple weeks.

But at that point, they were already so invested.

And We think that it was a more effective way of kind of gauging interest.

is related to the blogging idea that you guys did at Mufu.

Before launch, and very related to the question at the start, I was thinking, you've not talked about open source libraries.

Almost, I was thinking about it as a form of problem, but actually a usable open source library that people use, almost anybody can afford to use.

And then at some point figure out a premium with a charge model for that.

And that's the bottom part of what we're talking about.

So he's asking, is the open source model where you release a product out for people to start using for free as a way of building up an audience that you eventually will sell to.

Is that pretty much how that model works?

And so I think for most of the successful open source-based companies, say for GitLab, yes, that's pretty much how it works.

They're thinking of building up an audience.

I think for us, like that advice for us came directly from Jason Fried of 37 Signals.

So he had talked about this story at South by Southwest doing big things with small teams.

And they had been blogging for years on their blog 37 Signals before they finally launched Basecamp.

And they realized that was super valuable to them because people didn't start from 0 when evaluating their product.

They felt like they already had a relationship with these people, and they were all the exact kind of people, the audience, that they knew that they were building for.

The Segment story is pretty good in this way as well.

Yeah, so the founding of this kind of segment, the story around how segment got started, they actually created an open source library, analytics.js, that you can embed into your website, and it would kind of duplicate all of your analytics to, or it would send analytics from your site to all of the different existing kind of aggregators.

And it was an open source project, became like, just went crazy on Hacker News.

And they ended up using that as a way to then sell the rest of their offering from there.

I think it was like a crazy small number of lines in the first version.

And then the CEO at the time was like, not a fucking chance.

People on Hacker News just couldn't get enough of it.

People were asking, how do I get access to this code, etc.

It's a counterpoint to the charge money early because that was a free open source project that had no business model whatsoever at the beginning.

But again, got a bunch of users in there figure out, like, OK, how do I charge?

How do I turn this into something of value?

I'm talking about maybe product releases.

So especially with Apple, you have a hardware product.

Like, at what point is your MVPX, what is that other tool you're trying to build?

So for example, like, if you do get a paper wash, like that's obviously not a smart wash, but launch something and get feedback.

I love, I love talking about this because this is a problem that comes up all the time.

So especially for hardware companies, but I would abstract the problem, the question to like broader, like if your MVP actually takes a lot of work to do, if there's a lot of building that needs to happen before you can offer kind of that first modicum of value to users, how do you do that?

I love this because I think that there are great ways that you can hack around this problem.

It's very easy to say, oh, I'm building this super complex piece of software or hardware, I need to spend at least 2 years before I can talk to customers.

Or before I can find out whether they're interested in buying this.

You can start talking to customers on day 1 and here's how.

Talk about the existing ways that they're solving this problem.

Figure out what are the products they currently use to solve it and figure out what the pain points, what the problems are.

From there, the way that you validate that they're interested in buying something is hack together off-the-shelf existing technology that kind of approximates the problem.

Obviously it's not going to be as amazing as your beautifully designed perfect thing that you're working on, but you'll get an approximation of it.

Show that approximation, show that hack to the person who just described that they have a problem and try to sell it.

Dress up other people's technology as your own.

Like put it together, put your own 3D printed box on it, you know, duct tape it together, put your name on it, and try to sell it.

Because what I've learned is that if people aren't even interested in buying or using the hack together solution, then the thing that you're going to work on for 2 years may not actually be that much of a solution to the problem that makes it exciting.

Really, if you're working on a problem that's so burning, that's such an acute problem, the user should be willing to work with a crappy hack-together solution before you finish your real 1.

Edith Piaffi You can do so much with a landing page to just understand interest, whether your message, the idea resonates.

If they have such a burning problem, they see a landing page for this product that finally solves their wildest hopes and dreams.

I think the other thing is, Dropbox actually started that way.

So Dropbox was launched as an After Effects movie that approximated what the experience was going to be like, the shared Dropbox.

It's like, I want to know when that software is available.

And almost no lines of code had actually been written.

And so it was basically a product demo that launched that company.

Yes, for hardware consumer type of face nice products, how do you grow your pre-sales?

At that time it was a little bit easier, it's crowded now.

Instagram is very crowded now and it's very expensive.

Top of the funnel, ads, Facebook, Google ads, very expensive.

Are there any hacks around today's hacking ways?

So the question is, you know, it's harder to get a consumer product out there, like the digital space on Instagram, Facebook, everywhere, it's getting much more expensive and it's true.

I see a ton of like new products that are launching on Instagram and I keep thinking like how much money did they pay to get 1 ad in front of me.

So what can you do at an early stage without that much money?

See all Y Combinator transcripts on Youtube

Startup School Q&A Week 1