This is a clip from a conversation with Gustav Soderstrom from Jul 2019. New full episodes every Mon & Thu and 1-2 new clips or a new non-podcast video on all other days. You can watch the full conversation here: https://www.youtube.com/watch?v=v-9Mpe7NhkM
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on iTunes:
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a few clips every Tue & Fri. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Gustav Soderstrom is the Chief Research & Development Officer at Spotify, leading Product, Design, Data, Technology & Engineering teams.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

There's an interesting statistic I saw that, so Spotify has, maybe you can correct me, but over 50 million songs, tracks and over 3 billion playlists.

So 50 million songs and 3 billion playlists, 60 times more playlists than songs.

So, the way I think about it is that from a statistician or machine learning point of view, you have all these, if you want to think about reinforcement learning, you have this state space of all the tracks and you can take different journeys through this world.

And I think of these as people helping themselves and each other, creating interesting vectors through this space of tracks.

And then it's not so surprising that across, you know, many tens of millions of kind of atomic units, there will be billions of paths that make sense.

And we're probably pretty quite far away from having found all of them.

So kind of our job now is users, when Spotify started, it was really a search box that was for the time pretty powerful.

And then I like to refer to it as this programming language called playlisting, where if you, as you probably were pretty good at music, you knew your new releases, you knew your back catalog, you knew your Star Way to Heaven, you could create a soundtrack for yourself using this playlisting tool, that's like metaprogramming language for music, to soundtrack your life.

And people who were good at music, it's back to how do you scale the product.

For people who are good at music, that wasn't actually enough.

If you had the catalog and a good search tool, you can create your own sessions, you could create really good a soundtrack for your entire life.

Probably perfectly personalized because you did it yourself.

But the problem was most people, many people aren't that good at music.

Even if you're very good at music, it's going to be hard to keep up.

So what we did to try to scale this was to essentially try to build, you can think of them as agents that this this friend that some people had that helped them navigate this music catalog.

But also there is something like 200 million active users on Spotify.

So there, okay, so from the machine learning perspective, you have these 200 million people plus.

They're creating, it's really interesting to think of playlists as, I mean, I don't know if you meant it that way, but it's almost like a programming language.

It's a release, a trace of exploration of those individual agents, the listeners, and you have all this new tracks coming in.

So it's a fascinating space that is ripe for machine learning.

How can playlists be used as data in terms of machine learning and to help Spotify organize the music?

So we found in our data, not surprising that people who playlisted a lot, they retain much better.

And so our first attempt was to playlist for users.

And so we acquired this company called Tunigo of editors and professional playlisters and kind of leveraged the maximum of human intelligence to help build kind of these vectors through the track space for people.

Then the obvious next, and we used statistical means where they could see when they created a playlist, how did that playlist perform?

They could see skips of the songs, they could see how the songs perform, and they manually iterated the playlist to maximize performance for a large group of people.

But there were never enough editors to playlist for you personally.

So the promise of machine learning was to go from kind of group personalization, using editors and tools and statistics to individualization.

And then what's so interesting about the 3 billion playlists we have is we ended, the truth is we lucked out.

This was not a priority strategy as is often the case.

It looks really smart in hindsight, but it's dumb luck.

We looked at these playlists and we had some people in the company, a person named Eric Bernadon, who was really good at machine learning already back then, in like

Back then it was mostly collaborative filtering and so forth, but we realized that what this is, is people are grouping tracks for themselves that have some semantic meaning to them.

And then they actually label it with a playlist name as well.

So in a sense, people were grouping tracks along semantic dimensions and labeling them.

And so could you use that information to find that latent embedding?

And so we started playing around with collaborative filtering and we saw tremendous success with it, basically trying to extract some of these dimensions.

And if you think about it, it's not surprising at all.

It'd be quite surprising if playlists were actually random, if they had no semantic meaning.

For most people, they group these tracks for some reason.

So we just happened to cross this incredible data set where people are taking these tens of millions of tracks and grouped them along different semantic vectors.

And the semantics being outside the individual users, so it's some kind of universal.

There's a universal embedding that holds across people on this earth.

Yes, I do think that the embeddings you find are going to be reflective of the people who playlisted.

So if you have a lot of indie lovers who playlist, your embedding is going to perform better there.

But what we found was that, yes, There were these latent similarities.

And we had, it was interesting because I think that the people who playlisted the most initially were the so-called music aficionados who were really into music.

And they often had a certain, their taste was often geared towards a certain type of music.

And so what surprised us, if you look at the problem from the outside, you might expect that the algorithms would start performing best with mainstreamers first, because it somehow feels like an easier problem to solve mainstream taste than really particular taste.

The recommendations performed fantastically for people who saw themselves as having very unique taste.

That's probably because all of them playlisted and they didn't perform so well for mainstreamers.

They actually thought they were a bit too particular and unorthodox.

So we had the complete opposite of what we expected.

Success within the hardest problem first and then had to try to scale to more mainstream recommendations.


Speaker 0: There's an interesting statistic I saw that, so Spotify has, maybe you can correct me, but over 50 million songs, tracks and over 3 billion playlists. So 50 million songs and 3 billion playlists, 60 times more playlists than songs. What do you make of that?

Speaker 1: Yeah. So, the way I think about it is that from a statistician or machine learning point of view, you have all these, if you want to think about reinforcement learning, you have this state space of all the tracks and you can take different journeys through this world. And I think of these as people helping themselves and each other, creating interesting vectors through this space of tracks. And then it's not so surprising that across, you know, many tens of millions of kind of atomic units, there will be billions of paths that make sense. And we're probably pretty quite far away from having found all of them.

So kind of our job now is users, when Spotify started, it was really a search box that was for the time pretty powerful. And then I like to refer to it as this programming language called playlisting, where if you, as you probably were pretty good at music, you knew your new releases, you knew your back catalog, you knew your Star Way to Heaven, you could create a soundtrack for yourself using this playlisting tool, that's like metaprogramming language for music, to soundtrack your life. And people who were good at music, it's back to how do you scale the product. For people who are good at music, that wasn't actually enough. If you had the catalog and a good search tool, you can create your own sessions, you could create really good a soundtrack for your entire life.

Probably perfectly personalized because you did it yourself. But the problem was most people, many people aren't that good at music. They just can't spend the time. Even if you're very good at music, it's going to be hard to keep up. So what we did to try to scale this was to essentially try to build, you can think of them as agents that this this friend that some people had that helped them navigate this music catalog.

That's what we're trying to do for you.

Speaker 0: But also there is something like 200 million active users on Spotify. So there, okay, so from the machine learning perspective, you have these 200 million people plus. They're creating, it's really interesting to think of playlists as, I mean, I don't know if you meant it that way, but it's almost like a programming language. It's a release, a trace of exploration of those individual agents, the listeners, and you have all this new tracks coming in. So it's a fascinating space that is ripe for machine learning.

So is there is it possible? How can playlists be used as data in terms of machine learning and to help Spotify organize the music?

Speaker 1: So we found in our data, not surprising that people who playlisted a lot, they retain much better. They had a great experience. And so our first attempt was to playlist for users. And so we acquired this company called Tunigo of editors and professional playlisters and kind of leveraged the maximum of human intelligence to help build kind of these vectors through the track space for people. And that broadened the product.

Then the obvious next, and we used statistical means where they could see when they created a playlist, how did that playlist perform? They could see skips of the songs, they could see how the songs perform, and they manually iterated the playlist to maximize performance for a large group of people. But there were never enough editors to playlist for you personally. So the promise of machine learning was to go from kind of group personalization, using editors and tools and statistics to individualization. And then what's so interesting about the 3 billion playlists we have is we ended, the truth is we lucked out.

This was not a priority strategy as is often the case. It looks really smart in hindsight, but it's dumb luck. We looked at these playlists and we had some people in the company, a person named Eric Bernadon, who was really good at machine learning already back then, in like

Speaker 0: 2007, 2008.

Speaker 1: Back then it was mostly collaborative filtering and so forth, but we realized that what this is, is people are grouping tracks for themselves that have some semantic meaning to them. And then they actually label it with a playlist name as well. So in a sense, people were grouping tracks along semantic dimensions and labeling them. And so could you use that information to find that latent embedding? And so we started playing around with collaborative filtering and we saw tremendous success with it, basically trying to extract some of these dimensions.

And if you think about it, it's not surprising at all. It'd be quite surprising if playlists were actually random, if they had no semantic meaning. For most people, they group these tracks for some reason. So we just happened to cross this incredible data set where people are taking these tens of millions of tracks and grouped them along different semantic vectors.

Speaker 0: And the semantics being outside the individual users, so it's some kind of universal. There's a universal embedding that holds across people on this earth.

Speaker 1: Yes, I do think that the embeddings you find are going to be reflective of the people who playlisted. So if you have a lot of indie lovers who playlist, your embedding is going to perform better there. But what we found was that, yes, There were these latent similarities. They were very powerful. And we had, it was interesting because I think that the people who playlisted the most initially were the so-called music aficionados who were really into music.

And they often had a certain, their taste was often geared towards a certain type of music. And so what surprised us, if you look at the problem from the outside, you might expect that the algorithms would start performing best with mainstreamers first, because it somehow feels like an easier problem to solve mainstream taste than really particular taste. It was the complete opposite for us. The recommendations performed fantastically for people who saw themselves as having very unique taste. That's probably because all of them playlisted and they didn't perform so well for mainstreamers.

They actually thought they were a bit too particular and unorthodox. So we had the complete opposite of what we expected. Success within the hardest problem first and then had to try to scale to more mainstream recommendations.

See all Lex Fridman transcripts on Youtube

Machine Learning at Spotify - Gustav Soderstrom | AI Podcast Clips