- Emmett Shear, CEO of Twitch, discusses the importance of user interviews in starting a startup.
- He shares his own experiences with startups and how they lacked user feedback.
- Shear emphasizes the need to talk to users and shares how Twitch's success was driven by user interviews.
- He highlights the importance of identifying the target user for a product.
- Shear engages the audience in a practice exercise to think about who they would talk to for a lecture-focused note-taking app.

- Selling products to college students is challenging because they don't spend much money.
- College IT administrators may be more likely to buy a note-taking app for students.
- Other potential customers to consider are parents of college students and high school students.
- It is important to conduct user interviews to understand current note-taking habits and needs.
- Features of existing note-taking apps, such as Google Docs and Evernote, should be considered when developing a new app.
- The goal is to create a feature that would make the user switch from their current app to the new one.

- The speaker discusses the challenge of convincing someone to switch from Google Docs to a new collaborative tool.
- They suggest creating a mobile version of Google Drive that focuses on note-taking.
- The speaker emphasizes the importance of validating ideas before building them.
- They caution against asking users if they would be interested in a new feature, as their initial excitement may not translate to actual usage.
- The speaker suggests finding ways to hack together a prototype or getting users to commit to paying for the product to validate its appeal.
- They provide examples of feedback received during the development of Twitch, highlighting the importance of comparing feedback from different user groups.
- The speaker concludes by mentioning the importance of talking to non-users to gain further insights.

- The company talked to non-broadcasters to understand their needs and preferences
- They focused on non-users as they make up the majority of the market
- Feedback from non-users helped identify issues and goals that needed to be addressed
- The company invested resources in improving broadcasting technology and platforms
- They focused on addressing the concerns of non-users, such as making money, stability, and universal access
- The company found that user interviews and feedback were crucial in developing successful features
- Startups often make the mistake of showing their product instead of understanding what users already want
- It is important to talk to the right users, even if they are not easily accessible
- Recording interviews and playing them back for the company helps gain buy-in from the team
- Interviews should be done through Skype or in person for better interaction and insights
- International markets pose challenges in conducting user interviews due to language barriers.

- The channels used to reach out to users for interviews were on-site messaging systems and email, as well as meeting them at events and exchanging contact information.
- Users were not compensated for participating in interviews, as it was believed that if they were not willing to talk without payment, they may not be the right target audience.
- On-site user feedback tools were not heavily relied upon, as they focused more on data-driven approaches and ironing out kinks in existing products rather than discovering new problems to solve.
- The startup primarily focused on users of competing products, as they were already interested in the behavior needed and were more likely to switch to the new product.
- Initially, game publishers were not interested in talking to the startup, but as it gained traction, they became an increasingly important group to interview.
- It is important to continue conducting user interviews over time, as the pool of people who are crucial in the beginning may not be the same as those using the product in the long term.
- When giving user feedback, it is important to provide a full picture and talk about personal experiences and problems to help the interviewer understand why certain features are desired.

- Emmett Shear, CEO of Twitch, emphasizes the importance of user interviews in starting a startup.
- He discusses the need to talk to users and shares how Twitch's success was driven by user interviews.
- Shear highlights the importance of identifying the target user and conducting user interviews to understand their needs.
- He suggests considering features of existing apps and creating a feature that would make users switch to a new app.
- Shear mentions the importance of validating ideas before building them and gaining insights from non-users.
- The company invested resources in improving broadcasting technology and platforms based on user feedback.
- Startups should focus on understanding what users want instead of just showing their product.
- Recording interviews and conducting them through Skype or in person is recommended for better interaction and insights.
- International markets pose challenges in conducting user interviews due to language barriers.
- The startup primarily focused on users of competing products and continued conducting user interviews over time.
- When giving user feedback, it is important to provide a full picture and talk about personal experiences and problems.

Lecture Transcript: http://tech.genius.com/Emmett-shear-lecture-16-how-to-run-a-user-interview-annotated

Building product, and talking to users. In the early stages of your startup, those are the two things you should focus on. 
In this lecture, Emmett Shear, Founder and CEO of Justin.tv and Twitch, covers the latter. What can you learn by talking to users that you can’t learn by looking at data? What questions should you ask? How can user interviews define or redefine your product goals?

See the slides and readings at http://startupclass.samaltman.com/courses/lec16

Discuss this lecture: http://startupclass.co/courses/how-to-start-a-startup/lectures/64045

Chapters (Powered by https://bit.ly/chapterme-yc) - 
00:00 - Introduction
00:28 - First startup
01:07 - Second startup
04:44 - Talking to users
05:50 - Lecture focused note taking app
06:58 - Who would you talk to?
12:03 - User Interview
16:25 - Not interested in features
19:14 - One feature on top of Google Docs
22:44 - Validate the idea
24:08 - Hack something together
25:43 - Justin.tv Broadcaster Feedback
28:20 - Competitor Broadcaster Feedback
29:34 - Non-Broadcasters
29:45 - Non-Broadcaster Feedback
32:16 - Where were the issues?
34:49 - Audience question 
35:03 - The most common mistake
36:27 - How hard was it to get buy in from the rest of your company?
36:41 - How hard is it to get buy in from the rest of your company?
37:44 - Skype interviews for recording
38:50 - User interviews - Don't speak your language
39:50 - Channels used and do you compensate
40:59 - User Feedback
42:13 - Focus on a specific segment
43:35 - Game publishers
45:13 - How do you give good user feedback

Emmett is the CEO of Twitch, which was acquired by Amazon, where he now works.

And Emmett is going to do a new format of class today.

And talk about how to do great user interviews.

So this is the talking to users part of starting a startup.

Context, everyone knows where I'm coming from, from this.

We started our, I started my first startup with Justin Kahn right out of college.

We started this company called Kiko Calendar.

So that's not necessarily the end you want for your startup.

Neither of us were users of calendars, nor did we, during the period of time we worked on Kiko, talk to anyone who actually did use a calendar.

We got the build stuff part of the startup down.

The second startup we started, we used a very common trick that lets you get away with not talking to users, which is that we were our own consumer.

We we had this idea for a television show, Justin TV, a reality show about Justin Conn's life.

And we built a whole set of technology and website around the reality show we wanted to run.

And so we were the user for that for that product.

And that's actually 1 way to cheat and get away with not talking to many other users is, if you're just building something that literally is just for you, you don't need to talk to anyone else, because you know what it is you want and what you need.

But that's actually a really limiting way to start a startup.

Most startups are not just built for the person who is who is using them.

And when you do that, every now and then you get really lucky and you are representative of some huge class of people who all want the exact same thing you do.

But very often also that just turns into a side project that doesn't go anywhere.

So we kept working on Justin.TV for a while, and we actually achieved a good deal of success, because it turned out that there were people out there who wanted to do the same thing we did, which was broadcast ourselves live on the Internet.

But the issue with Justin.TV, the thing that, the thing that sort of kept us from achieving greatness is we hadn't figured out yet how to how to build towards anything beyond that initial TV show.

We knew how to, we built a great product actually.

If you wanted to run a live 24-7 reality television show about your life, we had the website for you.

But if we wanted to go do more than that, if we wanted to open it up to a broader spectrum of people, a broader spectrum of use cases, we didn't have we didn't have the insight to figure that out because we weren't that user.

And so at some point we decided to pivot Justin TV.

We decided we needed to go in a new direction.

We thought we'd built a lot of valuable technology, but we hadn't identified the use case that would let it get really big.

And there were 2 directions that seemed promising.

1 of them was mobile, and 1 of them was gaming.

And I led the gaming initiative inside of the company.

And what we did with gaming that was very, very different from what we'd ever done before was, we actually went and talked to users.

Because while I loved watching gaming video, I was very aware that neither I nor anyone else in the company knew anything about broadcasting video games.

And so I was amped about the content, I thought there was a market there, that was sort of the insight that that I had that I think wasn't common at the time, which is how much fun it was to watch video games.

Quick show of hands, people know about watching video games on the Internet here?

Okay, I'm just going to assume that people listening to this also know about it.

If you don't know about watching video games on the Internet, you should go read about that.

Because it's sort of important context for the stuff I'm going to talk about.

But the main point is I thought that was awesome but I didn't know anything about the side of it that was really important, which is actually acquiring the content to start broadcasting.

So we went out and We ran a, actually a very large number of user interviews.

We talked to a lot of people and brought that data back.

And that formed the core of all the decision making that was for the next 3 years of product features on Twitch.

Was sort of, I mean, the insights we got from that.

And we continued to talk to users and in fact built an entire part of the company whose job it is basically to talk to our users.

Which is a whole division that we just didn't even have at Justin TV.

We had no 1 at the company whose job it was to talk to our most important users.

And I'm gonna, I wanna give you guys a little bit of a, a little bit of an insight into with Twitch, what what that, what that meant going to go talk to users.

So we determined that the broadcasters were the most important people.

And the reason we determined that was when we went and looked into the market, we looked into what, what determined why people watched a certain streamer, or went to a certain website.

They would just follow the content, right?

You had a, you had a piece of content you loved and the broadcaster would come with you.

And that's actually the 1 really important point about user interviews, which is that who you talk to is as important as what questions you ask and what you pull away from it.

Because if you go and talk to a set of users, if we'd gone and talked to viewers only, we would have gotten a completely different set of feedback than talking to the broadcasters.

And talking to the broadcasters gave us insight into how to build something for them.

And that turned out to be strategically correct.

I wish I could tell you the recipe for figuring out who the target user is for your product and who your target user should be, but there isn't a recipe.

It comes down to think really hard and, and use your, use your judgment to figure out who you're really building this for.

So what I want to do is a little bit, something a little bit interactive now.

Which is, we're, I'm gonna, I got a bunch of ideas from, from you guys actually, so sort of suggested ideas.

And I want everyone to sort of sit down and do, do step 1 of this process for me right now.

Which is, think about, who would you go ask about this?

Where would you go to find the people you need to talk to about this in order to in order to learn about what you should build.

And so the idea we're going to use is, let me see here, of these ideas.

So here, it's a lecture focused note taking app.

The idea is, I don't think that the state of the art for note taking is good enough yet.

And I want to make a note taking app that improves that experience.

Or taking notes while listening to a lecture online better.

So, you know, maybe it has collaboration features.

Maybe it like helps you focus better somehow.

So, take like, take 120 seconds right now and think about not what you would ask or what the right features for this app is, but who would you talk to?

Who should the, who, Who's gonna give you that feedback that's gonna tell you whether this is good or not?

Take your laptop out, like type, write some stuff down.

Think, think about, like, the, you can, it's, it's good enough to, like, think that in your head.

But actually, like, if you actually just write it down and like just come up with the 5 people you'd talk to, the 5 types of people you'd talk to.

And who you think the most important 1 was.

Because there's nothing like actually running through a practice of something and trying to do it to actually get it into your head the right way to do it.

I'm gratified to hear clicking in the, of keyboards now.

If you're following along at home, pause, actually do it, think about who you, who would you talk to?

Because that's a, That is the first question for almost any startup that you need to answer is like who is my user and, and where am I gonna find them?

That's like way shorter than you'd normally use to think about this problem.

It's actually a really tricky problem and like figuring out where to source the people is pretty hard.

But we're going to move along anyways in the, in this highly abbreviated version of learning how to build a product and run a user interview.

So can can I, can I get 1 volunteer from the audience to come up and tell, tell us what who you would talk to and we'll talk about it?

I would definitely talk to college students First, obviously, because we sit in a lot of lectures.

And specifically, I want to talk to college students studying different subjects to see if maybe, you know, if you're an English major, if that makes a difference versus your studying math or computer science in terms of how you want to take notes in

And so you, so you're gonna talk to, talk to a bunch of college students.

Would you pick any particular subset of college students?

Like, sounds like you really, we're gonna talk to all college students, or like a broad array?

I wanna talk to college students like, And break down the divisions by like people who study different areas maybe.

And then also maybe it would make sense for people who have like different study techniques.

Cuz some people take a lot of notes, some people don't take that many notes, but still jot stuff down.

So I mean that, that's a really good start.

Like that's, that is actually obviously a group of users you want to go talk to, especially if you're targeting something at, you know, at college students as the consumer.

And if you're talking to college students as a consumer, the, you're going to get a lot out of students about what their current note taking habits are and, you know, what they would be excited about.

1 of the problems of selling things to college students is that college students don't actually spend very much money.

It's, it's really hard to get you guys to open your wallets, especially if you want them to pay for a school related thing.

I mean, people don't even want to buy textbooks, right?

I think you probably, probably all use Chegg or that, you know, borrow it from your friend or whatever.

And so, 1 of the things that I think you'd be missing if you go after just the students, right, is you want to figure out Who is the most important person to this, to this app?

And if you actually had a note taking app, my guess is for colleges, the people most likely to actually buy a note taking app that you guys would use would be college IT, right?

I mean, presumably for most, for the most part, if you want to sell software to students, like the people who have to get bought into that, is usually the school administrators.

So that would be not, that would be 1 approach.

If you thought, okay, well, you presumably go talk to the college students and you find out they don't actually buy any note taking software right now at all.

This is why you actually have to go talk to the users.

But you then have to try to maybe try other, other groups, right?

So I would talk to college, I would talk to IT administrators as well.

I think that's a, that's another area that's really promising.

Who, who, who spends money on their kids' education and is like willing to pull their wallet out?

Parents of kids who are freshmen or going off to college for the first time.

You need this app to make your kid productive so that they don't fail out of college.

And, and there's actually a lot of groups that are potential, that aren't necessarily the obvious user, but who are critical, critical to your app's success, potentially.

And you, when you're, when you're, at the very beginning of a startup like this, when you're like, you have this idea that you think is awesome.

You want to have that broadest group you possibly can.

You don't just want to talk to 1 type of person and, and, and learn that.

You want to get familiar with the various kinds of people who could be contributing.

So let's let, let's have someone come up and we're gonna pretend we're gonna run this user interview.

So we're gonna talk to a college student and try to find out what we should build.

You know, what we should get into this note taking app.

So, so some, another volunteer please for, for running an interview.

Welcome, thank you for agreeing to do this user interview with us.

So I wanted to hear from you about, you know, what are your note taking habits?

I like to now because of speed and efficiency and just to come back to it later, it's easy for me to just take notes on my laptop.

And so a lot of those notes will be primarily text based.

But in certain classes, so for example, if I'm taking a history class, most of it will be in text.

But if I'm taking a physics class, for example, there are gonna be more complex diagrams, different angles that I have to draw.

And so that's a little harder for me to get the.

What software do you use for this stuff today?

You take notes with your computer sometimes.

And when you take notes with, When you take all these notes at the end, do you actually review them?

Do you, be honest, do you actually go back and ever actually look at these notes?

But yes to the software base, because it's more easy to access and it's easier for me to share and collaborate and maybe even merge notes with other classmates and friends.

So what do you, what do you stick notes today on your computer?

And tell me more about like why 2 things at the same time.

So Evernote is easy if I'm trying to just collect it for myself, I think.

And yes, you can share, but I think Google Docs for me is easier to share.

And it depends also if, you know, a friend has already created a folder, for example, in Google Docs, and I just have to add to that folder if it's a group project, for example, versus if it's for my personal use, I, I tend to go more towards that.

So it sounds like you do a lot of like, note taking collaboration.

What tell me more about that, like, like, do you take a, do you wind up taking most of the notes, most of the value of the notes out of notes other people take?

Or is it mostly your own notes you review at the end of the semester?

It's mostly mine because I'm pretty picky about the way I like things organized.

Like design wise or formatting even color.

I'm really particular with and like the font that we use, and that really affects the way I study.

So I tend to like it, to like, to like to personalize it, even after I merge.

So you, you, you pull in notes from other people, But then you merge them into, into the main.

And if you if you have Evernote notes and you have Google Docs notes and you have pen and paper notes.

Once the semester's over, do you ever go back to any of that stuff?

Or is it like, or quarter, you guys are in quarters here, right?

Once the quarter is over, do you ever go back to any of that stuff?

But if it's notes that I've taken for talks like these, for example, or if it's interview prep that I'm doing, I tend to go back because it's things that I like to kind of keep fresh in my mind and to help me prep for future things.

So I take notes to also just summarize main points.

So if it's like inspirational quotes, for example, from talks that I go to like these.

And then like maybe I'm going to an event where I'm actually going to meet someone.

And it, it helps to actually to think about and to remember and recall what was shared at the time that, you know, I attended the talk or something.

All right, well, normally I'd actually dig into a lot more detail.

There's a huge amount of like, open questions that are still in my mind after hearing that stuff.

Questions about which people do you collaborate with?

Questions about whether or not you, like, like what the volume of notes are and like how long of note taking stuff.

And just sort of digging into like what the current behavior is.

But like in the interest of time and not like keeping everyone here hearing about the intricacies of 1 person's note taking habits forever we're going to move on.

You notice we're not talking about the actual content of the app at all.

Like I'm not, I'm not, I'm not really interested in features.

I don't really want to know about what they, the specific feature set in Google Docs or Evernote, I might start digging in a little bit more into which features actually get used.

Like, if she's actively collaborating, you know, how does that work?

I heard some interesting things about, oh, we have, we use folders.

But the main thing you're trying to do when you're running these first set of interviews is not necessarily get, like, questions about, like, user flows and, like, optimizing that, or questions about, like, the specifics of, of, of any of that stuff kind of can be distracting.

Because users think they know what they want.

But like, you, you get the you get the horseless carriage effect where you're, you're you're, you're getting asked for a faster horse instead of trying to design the actual real solutions to the problem.

If you start asking people about features.

You want to stay as far away from features as possible.

Because the things they tell you wind up almost feeling overwhelmingly real.

When you have a real user asking you for a feature, it's almost, it's very hard to say no to them.

Because here's a real person who really has this problem, and they're saying build me this feature.

But as you start to talk to lots of people and really get a sense for what their problems are, you figure out if this is actually a promising area or not.

And like, based on what I heard there, it's like starting from that user interview, I'm not necessarily positive there is a problem.

Or there's at least, there's a big enough problem that it's worth building a whole new product for.

Because I didn't hear a lot of like, things where I'm, where I, where there was a, a big blocker, where there's something really wrong with the way, the way it it was working.

And unless I had some big idea I would take that as a, you know, maybe a negative sign.

But it doesn't necessarily mean that you can't, you can't move forward and keep talking to more people.

Because just because you talk to the first person and you don't get anything out of it doesn't mean there's not going to be a ton more people who actually have a problem.

And you, once you've talked to about 678 people you're usually about done.

It's unlikely you're going to discover a bunch of new information there.

Which is why it's important to talk to different extremes of people, right?

Go, go find people who are at different, different points because if you talk to 6 or 7 Stanford college students, you're going to get a very different response than if you talk to 6 or 7 high school students or 6 or 7, 6 or 7 parents.

I think the, I think it's possible you could come up with a set of ideas, right?

You have this information about how someone takes notes.

You've, you've come up with potentially when you came up with this idea you had, you had some ideas as to, when you heard this idea you had some ideas as to like, how you could build something cool.

And so, if you're gonna build just 1 feature on top of Google Docs, what would that feature be, right?

And that's, for, for, for a new product like this, it might be a good way to like get started thinking about where to go, which is okay, so they're extensively using this thing right now.

How can we make that experience just 1 quantum better, something that would be really exciting to this person, to be 1 1 step ahead.

And so, I want you to take 2 minutes right now and think about what that feature might be.

Actually like, try to, try to come up with what what you might do based on what you heard from from Stephanie that could convince her to switch away from her current collaborative, multi-person, all working together workflow on Google Docs to your new, your new thing that has all the features of Google Docs plus this 1 special thing that's like going to make it that's going to make it more more useful and convince them to stop using the thing they're already using.

So I'm gonna invite our, our third guest.

If you, if you, if you have something up.

I, I don't want to put you on the spot if you feel like you, you're not sure, but yeah?

What I thought about was like, the, the reason she uses Evernote is because like, of like sticky note type notes, like, like more thoughts and like details.

See all Y Combinator transcripts on Youtube

How to Run a User Interview with Emmett Shear (How to Start a Startup 2014: Lecture 16)