My guest today is Des Traynor. Des is the co-founder and Chief Strategy Officer of Intercom - a customer service solution that helps businesses answer product questions, offer instant support, and automate sales. The business was founded in 2011 and its products operate within 25,000 businesses, including the likes of Amazon, Lyft and Atlassian. Our conversation is roughly split in half. First, we talk about AI and how it’s actually changing businesses like Intercom through products such as their OpenAI powered bot called Fin. We then talk about Des’s views as an investor, which includes an answer about software that I’ll remember for a long time. Please enjoy my conversation with Des Traynor.

Listen to Founders Podcast
Founders Episode 311: James Cameron

For the full show notes, transcript, and links to mentioned content, check out the episode page here.

-----

This episode is brought to you by Tegus. Tegus is the modern research platform for leading investors, and provider of Canalyst. Tired of calculating fully-diluted shares outstanding? Access every publicly-reported datapoint and industry-specific KPI through their database of over 4,000 drivable global models hand-built by a team of sector-focused analysts, 35+ industry comp sheets, and Excel add-ins that let you use their industry-leading data in your own spreadsheets. Tegus’ models automatically update each quarter, including hard to calculate KPIs like stock-based compensation and organic growth rates, empowering investors to bypass the friction of sourcing, building and updating models. Make efficiency your competitive advantage and take back your time today. As a listener, you can trial Canalyst by Tegus for free by visiting tegus.co/patrick.

-----

Invest Like the Best is a property of Colossus, LLC. For more episodes of Invest Like the Best, visit joincolossus.com/episodes. 

Past guests include Tobi Lutke, Kevin Systrom, Mike Krieger, John Collison, Kat Cole, Marc Andreessen, Matthew Ball, Bill Gurley, Anu Hariharan, Ben Thompson, and many more.

Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week. Sign up here.

Follow us on Twitter: @patrick_oshag | @JoinColossus

Show Notes	
(00:03:42) - (First question) - Des's high level thoughts on Open AI and new technology
(00:07:07) - Advanced AI tech for existing businesses versus start-ups ventures 
(00:11:14) - Where Intercom falls on the spectrum of existing and new AI technology 
(00:13:16) - Training AI to be tailored to specific fields of work
(00:18:25) - The natural end point is for Intercom incorporating AI  
(00:22:25) - Des's product philosophy behind this technology
(00:24:03) - Choosing an AI provider that best suited his customer service industry needs
(00:26:55) - The value comparison between using AI-led customer service versus human
(00:29:18) - Why outsourcing automated data is not for everyone
(00:31:15) - Des's advice for other companies beginning to integrate AI into their operations  
(00:36:33) - What he is excited for as an investor in this area
(00:37:50) - The most common discussions he’s having about this technology
(00:41:21) - The inherent risks of using AI models that are not 100% accurate 
(00:45:10) - Des's evolution as an investor
(00:48:25) - How Intercom solved it’s AI execution problems
(00:50:46) - Combating other companies that are trying to overtake the field 
(00:53:44) - Counter arguments for using AI technology  
(00:57:22) - His view on the best software out there
(01:03:00) - The kindest thing anyone has ever done for him 

If you're a regular listener of my podcast, I have another recommendation for you.

The podcast is called Founders, and it's hosted by David Senra, who has devoted his life to learning from history's greatest entrepreneurs.

Each week, David distills the lessons of a different founder, from Henry Ford to Coco Chanel to Edwin Land.

His recent episode on James Cameron is particularly excellent if you're looking for somewhere to start.

I guarantee if you listen to Founders, you'll feel inspired to level up whatever you're working on.

You can find a link to Founders in the show notes of this conversation.

And because Founders is part of the Colossus Network, you can search all the past transcripts on our website at joincolossus.com.

This episode is brought to you by Tegas, the modern research platform for leading investors and the provider of Catalyst.

Tired of calculating fully diluted shares outstanding?

Access every publicly reported data point and industry specific KPI through their database of over 4, 000 drivable global models, hand-built by a team of sector focused analysts, 35 plus industry comp sheets, and Excel add-ins that let you use their industry-leading data in your own spreadsheets.

Tegas's models automatically update each quarter, including hard-to-calculate KPIs like stock-based comp and organic growth rates, empowering investors to bypass the friction of sourcing, building, and updating models.

Make efficiency your competitive advantage and take back your time today.

As a listener, you can trial Catalyst by TGIS for free by visiting tgis.co.patrick.

I'm Patrick O'Shaughnessy and this is Invest Like the Best.

This show is an open-ended exploration of markets, ideas, stories, and strategies that will help you better invest both your time and your money.

Invest Like the Best is part of the Colossus family of podcasts, and you can access all our podcasts, including edited transcripts, show notes, and other resources to keep learning at joincolossus.com.

Patrick O'Shaughnessy is the CEO and founding partner of Positive Sum and the CEO of O'Shaughnessy Asset Management.

All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of Positive Sum or O'Shaughnessy Asset Management.

This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions.

Clients of Positive Sum or O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

Des is the co-founder and chief strategy officer of Intercom, a customer service solution that helps businesses answer product questions, offer instant support, and automate sales.

The business was founded in 2011 and their products operate with 25, 000 businesses, including the likes of Amazon, Lyft, and Atlassian.

Our conversation is roughly split in half.

First, we talk about AI and how it's actually changing businesses like Intercom through products such as their open AI powered bot called FIN.

We then talk about DES's views as an investor, which includes an answer about software that I'll remember for a long time.

Please enjoy my conversation with DES Trainer.

I've been so excited for this because I think it's the perfect excuse to talk to a real practitioner, builder, entrepreneur about what it's been like to feel the explosion of AI and its impact, both exciting, maybe even scary, on an existing product, an existing software product that's always been very cutting edge in terms of its architecture, et cetera, and how you approach this new opportunity slash threat.

Everyone, I think, is bomb that went off in the technology scene.

And I'd love you to begin by just giving me your high level thoughts on what it's been like to see chat GPT, GPT-4, all the open source models, all this explosion of new technology through the lens of an existing really successful software business.

I think the world changed with chat GPT, both capabilities of what AI could do change.

And also I think people's willingness to engage with a bot was a lot different after their first experience of chat GPT.

For us, I met with our head of ML, I think the day after chat GPT dropped and we had a pretty long conversation where he was pretty firm that this is the single biggest transformation he's seen in AI and he's spent his entire career in it.

The more we played with what we realized was, Hey, this thing is very conversational.

It can learn, it can summarize, it can extract the main pieces and given a query and given a set of information, it can actually propose an answer.

And that is basically what customer support is.

So the implications for customer support and intercoms were customer service platform, the implications are pretty obvious.

It sounded probably more like a prophecy at the time, but I think looking back, everyone's like, yeah, no, it's pretty obvious that the entire world of customer support was going to change.

So the question was, how quickly can we do it?

It wasn't quite code red, but it was as close to a can be from an offensive point of view.

The team worked insanely hard and we produced our first release.

Well, we had our first release in beta in December.

And then we talked about it publicly in January.

And that was all about assisting the customer support rep, because we weren't yet sure that the tech would actually be good enough to face end users, but we knew it was definitely good enough to augment the customer support agent.

As soon as we got our hands on that, that was where we were able to say, Hey, we can constrain this agent to not go off topic, to not take opinions about whatever politics or to not recommend your competitors or to not do anything you wouldn't want a support rep to do.

And we can also, we believe, curtail the vast majority of the hallucinations by basically with the right amount of prompting.

Then we went to general availability, I think in late May or early June.

Just seeing how the bot almost always is outperforming what our expectations of it are in terms of like the complexity, the nuance, customers will send us 7 questions, not knowing they're talking to a bot, 7 questions would nested.

Genuine conversations that might've taken a support rep an hour to aggregate all the information for Fin is just blitzing it.

We've seen customers see 50% of their support volume drop.

We've seen on average, most customers who turn Finn on with no other work, literally clicking an on button are getting

of their support volume just going away straight away.

And it's just been crazy to see a product where we knew we had done all the right stuff on our side, but you're still crossing your fingers going, I hope it works well in the aviation industry.

So it's been wild to see it up front And we've continued, and obviously we can talk more about it, but yeah, we're now asking ourselves what's the next tier of this?

How do we make it interoperate with humans better, et cetera.

Do you think that this technology is actually more powerful for existing companies that have prebuilt products and big data stores and big teams, et cetera, versus de novo startups that are trying to, let's say for example, someone wanted to build a from scratch intercom competitor starting tomorrow, And they've got all this great new technology often in the history of technology, if you're using new, you can counter position using new technological platforms or whatever, and really stick it to incumbents.

My experience so far has been, it feels different in this context than that.

Actually fast moving incumbents, as long as they recognize it are better positioned.

Any big thoughts that you have on that idea?

I've been thinking about this a lot, both from an intercom perspective, and there are a lot of people trying to do customer support startups based on open AI as tech at the moment.

And then I've also been thinking about as an investor as well, just looking at the startup scene, seeing what's new.

The way I've concluded is, and by the way, in your example, let's just assume we'll take off the table, the idea that GPT can write the code to make it at the intercom competitor.

Like let's assume all those things are true for everyone.

I think if the way in which you would rebuild the competitor, let's say I'm trying to build a competitor to intercom to make it not personal for lack of a better word, let's pick say MailChimp or something like that.

An email newsletter tool, everyone understands the gist of it.

The question I would have is does the way in which you'd build the competitor, is it substantially different to how MailChimp is architected?

Are there some fundamental assumptions made in the code base of MailChimp that are now entirely invalidated.

And if that is the case, then I think, yeah, the opportunity is with the new startup because what MailChimp has to do if they want to compete is actually build a whole new thing.

But I think in most cases, and if we just keep pulling the email MailChimp example, if you and I say, all right, hey Patrick, let's go build a MailChimp competitor and we're going to use AI and it's going to do AI to write your newsletters and AI to generate your designs.

So we still have to build a massive deliverability platform.

We still have to build email rendering and testing email rendering across a dozen different clients.

We still probably have to build up a brand credibility and all of that stuff.

And all MailChimp has to do is shout out to OpenAI for augmenting the text you're writing.

So in that world, I think you might be in some 80, 20, 80% of the existing text still stands and 20% is going to be new stuff coming from OpenAI or coming from Anthropic or whoever we lean on.

And the calculus I'd actually do when I talk to the companies about prospective investments, or when I assess intercom threats is basically how fast can they move, let's assume that you and me and our super hot new startup can move 10 X to speed of MailChimp.

And let's say we conclude it takes us 3 months to build all the AI stuff.

The question is, is 27 months enough for me and you to go and build every other feature in MailChimp to the same standard?

We don't really have a play to make there.

To give you just 1 counter example, let's say me and you said, hey, we're gonna build a tool that, it's 1 of these advertisement management optimization tools.

You log in every day and you see what ads are working and you change your spends and you cancel some ads and tweak some ads.

You could imagine how a LLM-powered tool could basically optimize itself, consistently create new versions of ads, run those instead, run A-B tests amongst itself, and literally entirely manage your entire ad inventory directly without anyone ever having to log in.

And in that world, I think a lot of the assumptions of the incumbent are totally invalid.

They might have dashboards and reports and lovely funnels for configuring occupants, but it's all unnecessary.

So in that world, if we're going to go after a space where, Hey, we would actually build this thing substantially different where we're building it today.

I think all the advantage is within the entrant.

So talking again about intercom, if with stats up to 50% of customer service requests is handled end to end without a human in the loop, the whole thing you just described is a gradient.

It's never 1 or the other, it's somewhere on a spectrum.

So Where do you feel like you fall on that spectrum of this is an entirely new thing versus, oh yeah, to build the rails underneath the delivery of this thing is like the MailChimp example, it takes 4 years.

I think we're somewhere in the middle where whole workflows are removed in Intercom's case, but not the entire platform.

There are still humans doing support and they still need a pretty rich and powerful support help desk and they still need a messenger to communicate through.

All of that needs to be available through APIs.

There's a lot of other stuff that still has to be built.

And then obviously Intercom also has proactive support, so there's messaging pieces as well.

So I think what we have had to do is reimagine and throw away parts of Intercom that assumed, say, our reporting infrastructure is very different in a world where 50% of the responses are going through AI.

All of a sudden people care a lot about the AI reporting, whereas they didn't before.

Measures like first response time are now no longer really valid because you have to subtract that AI and all that stuff.

So I think if you can imagine of all the support work that happens, X percent of it's gone, the remainder still needs what we would call a classical high quality help desk.

A good chunk of the work just disappears entirely.

And then there are some features or some workflows, we'll see how we play it out where we'll probably heavily augment them with AI.

So you could imagine analyzing What are the most common complaints from customers today?

I can imagine we'll throw a lot more AI at that feature to remove the needle in a haystack approach and actually maybe the new version of that report is just a summary paragraph that tells you here's the biggest issues going on today.

So I think that would be an example of workflow displacement.

But I think in general, we believe the future of customer service will still involve humans and bots.

And we care a lot about making sure that they can work together really well, and they can interoperate.

And we have this idea of a flywheel where the humans help the bots and the bots help the humans.

If we're right, anyone who wants to beat us also needs to have a pretty high quality help desk too.

Can you help me understand the nuts and bolts of working backwards from a thing that I think everyone wants, and I'll call that thing an agent that is context specific to their business or themselves or whatever, where the feeling of chat GPT, which is so magical or GPT4 is so magical, but with all the context and the knowledge trained on me or trained on my business.

And what you just said, people helping bots, bots helping people.

Sounds like this is a process to take the generic broad artificial intelligence, whichever provider you use, and make it context specific or really helpful to me specifically.

There's this process of retraining it or narrowly training it.

So can you just literally tell me the steps of what you've learned about how that works?

Because it seems reasonable to me that we're going in that direction.

Everyone's going to have an agent that is tailored and aware of their specific circumstances and context and data and so on.

And you're 1 of the first to actually build this process of training this thing.

And I think of Finn as okay, Finn's the intercom agent.

It is a multi-purpose thing, but it's very context specific to your product and your world.

So what have been the steps, the mistakes, the considerations, the details of this are really interesting to me if this is where the world's going.

There's a lot to this, but the biggest things I think we lean on OpenAI mostly to power Finns decision-making ultimately, like it's judgment and also it's conversational capability.

So we don't have to build stuff like, hi, how are you?

All of that type of boilerplate salutation and all that open AI is just really good.

And it's really good even when we want to get to a point where our customers can have Finns speak in their own brand and tone of voice.

So like a surf shop and a bank can sound appropriate for their domain.

So I think our biggest challenge initially was how do we get it to stay on topic and how do we get it to not apply knowledge that has nothing to do with your business.

So if you go to any finance instance and ask it like, who is the president of America?

It actually won't attempt to answer it because there's a slippery slope when you'll see a lot of people go down the slippery slope where you can force it to then take opinions that the company doesn't want to take.

Getting the bot, despite all its infinite wisdom, to refuse to engage in topics that are anything other than this bank or this surf shop or this podcast or whatever, it's an important step.

How would you say, please the user by making shit up there, the first 2 things in practice, how does that happen?

Well, we discern the most important pieces from the user.

So when they come along and say, Hey, how do I, whatever, reset my password or open an account or whatever, they look for the guts of what they're actually asking.

You perform some vector searches across all the docs you've gathered.

And that can include everything from your public knowledge base, the entire conversation that's led up until this point in the thread.

So it can do things like, how do I do that?

And it can infer what the data means, et cetera.

So you have to perform a search of, given that this is the question, drinking in all of this context, what do we suppose is the right answer?

And then how do we package that answer in a way that sounds again, on brand and appropriate for where the conversation is at right now, depending on the tone it's taking.

It's about the vector search is really important.

The knowledge sources are the single biggest variant.

When we see people like amazing Fin instances and like weaker Fin instances, it's literally how much have you fed it?

How much have you given it access to everything about this user, everything about your product, your product docs?

It's been impressive how Fin can even read stuff we didn't expect, be it like API docs or things like that, to actually suggest extra answers or further reading for the users.

But that's a lot of what it's been about.

And for us, 1 of our challenges was coming up with a ability to benchmark where different versions are at because We can tweak and change prompting or we can change model and point to the GPT 3.5 turbo versus 4.

And it's nuanced a lot of the time to see the differences until you spot them.

And then you realize, oh, this thing is actually going to go and recommend our competitor or it'll go off topic or whatever.

1 thing we've learned of late is there are so many people who have built their wrapper around GPT-3, 5 turbo.

And if you just test them all by asking a really obvious question and getting a really obvious answer, you're going to conclude they're all the same bot.

And I say unfortunate for us because it's in the nuance and in the more likely scenarios, that's where you realize some bots are very good and some bots are very bad.

It's really when you get into specifics about like a refund request, or you try and make it go off topic.

That's where you see it's under performance or you see the lack of prompting, the lack of training, et cetera.

And even yesterday we released a bot buyer's guide because we're trying to teach customers about these differences.

Obviously you hate having to market nuance.

We'd rather we were the only fin in town, but that's not the case.

It was the case for 3 or 4 months, but now there's a lot of YC startups doing it.

Now we're into who's got the most, who's taught this true to most.

Ultimately, I still think though, as I said earlier, the battle will really shift being who has the right platform, who has the holistic solution for customer service.

If you think about the examples you've seen so far, some of which have crazy staggering numbers that you quoted in terms of how much of the previous product that was human focused is now and then handled by a machine.

Where do you think the natural endpoint of this is for your business?

I'm curious what the adoption's been, like how many of the normal intercom customers are using FIN in some way, shape or form.

So what's the friction to getting on this train and where does it go?

Is the natural endpoint that customer service is 80, 90% handled by AI.

And it's just a strange edge cases get spit out to a much smaller support desk.

Where do you think this goes and what have been the frictions to adoption?

The largest barrier is the people aspect.

So it's customer support, generally speaking as a human operated industry.

And if we move a button in intercoms inbox, we get our support team just get fed fire by our customers.

Cause they're like, yo, if you're going to make a single change to this inbox, we have to have an offsite to retrain our entire staff.

And when you realize that that could be hundreds of people, we have customers who have thousands of intercom seats.

So 1 single change of, Hey, it used to say send and close.

That could literally set large intercom instance back weeks in terms of support volume.

So we're very delicate about how we make these changes.

The other side of that friction is our customers are very, very slow to adopt for a very good reason because they say, OK, we will try.

So we're seeing a lot in, say, Finn's case.

We have many, many, many hundreds, if not thousands of Finn users spitting out tens of thousands of answers on a regular basis, et cetera.

But what we're seeing is everyone wants to dip their toe.

They don't want to say let's point Finn at the entire support volume.

What they say is, Hey, let's turn Finn on, on the weekends.

Or they say things like let's turn Finn on if, and only if the question regards resetting a password or something like that.

And they're doing that because they want to get a sense of how is it performing in a smaller use case.

And then ultimately we've only really been live to literally everyone, I think about 8 weeks.

We have a lot of planned, larger migrations where our customer support teams are going to switch over to a massive amount of volume going to Finn.

But they're busy working on their knowledge bases and they're busy working on their snippets, et cetera.

The snippets being the things that Finn reads to produce answers.

So I think what we're seeing is a lot of people preparing for this world, but the friction is definitely how do I get my support team on board?

How do I make sure that we're doing all the right stuff?

Even in a lot of cases, all our help docs are out of date and we didn't realize it, but our customer support team were saving our ass and now we need to update our help docs before we let Finn in.

The other question we ask is where does it stop?

And I think that's genuinely something we don't really know the answer to.

We will very shortly have live in market this Finn generated snippets feature where Finn will read your entire conversational contacts for your business going all the way back.

And it'll learn from every live conversation that happens and it'll produce snippets, little nuggets of knowledge to augment its own understanding of your business along the way.

Our vision for that will be that your customer support team deals with any common query they see.

They should see it for the first time and the last time.

They should basically answer it once and believe that they'll never see it again.

Where we want them to spend their time is on high value, brand building, high urgency, high impact conversations.

We want them spending time in proactive support.

We want them reaching out rather than like dealing with customers when stuff goes wrong, we want them reaching out to make sure that everything goes right.

That's the future world we want for support.

In terms of what percentage could actually of raw inbound could go true, it will vary vertical by vertical.

If you think about, say, an e-commerce store, there's really only 10 questions you ever ask.

What's most exciting to us is what will this snippets feature do?

Because you're just going to keep aggregating knowledge.

If you take intercom, we do 20, 000 support conversations a month.

See all Invest Like the Best with Patrick O'Shaughnessy transcripts on Applepodcasts

Des Traynor - Real Talk about AI and Software