Gustav Soderstrom: Spotify
嘉宾：Gustav Soderstrom
全文文稿 · 按对话顺序保留完整内容，深入理解上下文。
§01Transcript · 00:00–13:25
The following is a conversation with Gustav Sørenstrøm.
He's the Chief Research and Development Officer at Spotify,
leading their product design, data technology, and engineering teams.
As I've said before, in my research and in life in general,
I love music, listening to it and creating it,
and using technology, especially personalization through machine learning,
to enrich the music discovery and listening experience.
That is what Spotify has been doing for years, continually innovating,
defining how we experience music as a society in a digital age.
That's what Gustav and I talk about among many other topics,
including our shared appreciation of the movie "True Romance,"
in my view, one of the great movies of all time.
This is the Artificial Intelligence Podcast.
If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support on
Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D-M-A-N.
And now, here's my conversation with Gustav Sørenstrøm.
Spotify has over 50 million songs in its catalog, so
let me ask the all-important question. I feel like you're the right person to ask.
What is the definitive greatest song of all time?
It varies for me, personally. So you can't speak definitively for everyone?
I wouldn't believe very much in machine learning,
if I did, right? Because everyone had the same taste.
So for you, what is... you have to pick. What is the song?
All right, so it's pretty easy for me. There is this song called
"You're So Cool" by Hans Zimmer, soundtrack to "True Romance."
It was a movie that made a big impression on me, and it's kind of been
following me through my life. Actually, I had it play at my wedding. I sat with the
organist and helped him play it on an organ, which
was a pretty interesting experience. That is probably my,
I would say, top three movie of all time. Yeah, this is an incredible movie.
And it came out during my formative years, and
as I've discovered in music, you shape your music taste
during those years. So it definitely affected me quite a bit.
Did it affect you in any other kind of way? Well, the movie itself affected me
back then. It was a big part of culture. I didn't really adopt any characters
from the movie, but it was a great story of love,
fantastic actors, and really, I didn't even know who Hans Zimmer was at
the time, but fantastic music. And so
that song has followed me, and the movie actually has followed me throughout my life.
That was Quentin Tarantino, actually, I think, director of
"Produce The Hatter". So it's not "Stairway to Heaven" or "Bohemian
Rhapsody". Those are great. They're not my personal
favorites, but I've realized that people have different tastes, and
that's a big part of what we do. Well, for me, I would have to
stick with "Stairway to Heaven". So, 35,000 years ago, I looked this up on
Wikipedia. Flute-like instruments started being used in caves
as part of hunting rituals, in primitive cultural gatherings, things like that.
This is the birth of music. Since then, we had a few folks, Beethoven,
Elvis, Beatles, Justin Bieber, of course, Drake.
So, in your view, let's start high-level philosophical. What is the
purpose of music on this planet of ours? I think music has many different
purposes. I think there's certainly a big purpose, which is the same as
much of entertainment, which is escapism, and to be able to live in some sort of
other mental state for a while. But I also think you have the opposite
of escaping, which is to help you focus on something you are actually doing.
So, I think people use music as a tool to
tune the brain to the activities that they are actually doing.
And it's kind of like, in one sense, maybe it's the rawest signal. If you
think about the brain as neural networks, it's maybe the most efficient hack we
can do to actually actively tune it into some state that you want to be. You
can do it in other ways. You can tell stories to put people in a certain mood.
But music is probably very effective to get you to a certain mood very fast.
You know, there's a social component historically to music, where
people listen to music together. I was just thinking about this, that
to me, and you mentioned machine learning, but to me
personally, music is a really private thing.
I'm speaking for myself. I listen to music.
Almost nobody knows the kind of things I have in my library,
except people who are really close to me, and they really only know
a certain percentage. There's some weird stuff that I'm almost probably
embarrassed by. It's called the guilty pleasures, right?
Everyone has that. The guilty pleasures, yeah.
Hopefully they're not too bad. For me, it's personal. Do you think of
music as something that's social or as something that's personal?
Or does it vary? I think it's the same answer, that
you use it for both. We've thought a lot about this
during these 10 years at Spotify, obviously. In one sense, as you said, music
is incredibly social. You go to concerts and so forth.
On the other hand, it is your escape, and everyone
has these things that are very personal to them.
What we've found is that when it comes to...
Most people claim that they have a friend or two that they are heavily
inspired by, and that they listen to. I actually think
music is very social, but in a smaller group setting, it's an
intimate relationship. It's not something that you
necessarily share broadly. Now, at concerts, you can argue you do,
but then you've gathered a lot of people that you have something in common with.
I think this broadcast sharing of music is something we
tried on social networks and so forth, but
it turns out that people aren't super interested in
what their friends listen to. They're interested in
understanding if they have something in common, perhaps, with a friend, but not
just as information. Right, that's really
interesting. I was just thinking of it this morning,
listening to Spotify. I really have a pretty intimate
relationship with Spotify, with my playlists. I've had them for
many years now, and they've grown with me together.
There's an intimate relationship you have with a library of music
that you've developed, and we'll talk about different ways we can play with that.
Can you do the impossible task and try to
give a history of music listening from your perspective, from before the
internet and after the internet, and just kind of everything leading up
to streaming with Spotify and so on? I'll try. It could be a 100-year podcast.
I'll try to do a brief version. There are some things that
I think are very interesting during the history of music, which is that
before recorded music, to be able to enjoy music, you actually had to be
where the music was produced, because you couldn't record it
and time shift it. Creation and consumption had to happen at the same
time, basically concerts. So you either had to get to the
nearest village to listen to music, and while that was
cumbersome and it severely limited the distribution of music,
it also had some different qualities, which was that
the creator could always interact with the audience. It was always live.
And also there was no time cap on the music. So I think it's not a coincidence
that these early classical works, they're much longer than
the three minutes. The three minutes came in as a
restriction of the first wax disc that could only contain
a three-minute song on one side, right? So
actually the recorded music severely limited the
or put constraints, I won't say limit, I mean constraints are often good, but it
put very hard constraints on the music
format. So you kind of said like instead of doing these opus
on like many, you know, tens of minutes or something,
now you get three and a half minutes because then you're out of wax on this
disc. But in return, you get an amazing
distribution. Your reach will widen, right?
Just on that point real quick, without the mass
scale distribution, there's a scarcity component
where you kind of look forward to it. We had that, it's like the Netflix
versus HBO Game of Thrones, you like wait for the event because you
can't really listen to it. So you like look forward to it and then
it's, you derive perhaps more pleasure because
it's more rare for you to listen to a particular piece.
You think there's value to that scarcity? Yeah,
I think that that is definitely a thing and there's always this
component of if you have something in infinite amounts, will you value it
as much? Probably not. Humanity is always seeking some,
is relative, so you're always seeking something you didn't have and when you
have it, you don't appreciate it as much. So
I think that's probably true, but I think that's why concerts exist, so you can
actually have both. But I think net, if you couldn't listen
to music in your car driving, that'd be worse,
that cost would be bigger than the benefit of the anticipation I think
that you would have. So yeah, it started with live concerts,
then it's being able to, you know, the phonograph
invented, right? You start to be able to record music.
Exactly, so then you got this massive distribution that made it possible
to create two things, I think. First of all, cultural
phenomenons, they probably need distribution
to be able to happen, but it also opened
access to, you know, for a new kind of artist. So you started to have these
phenomenons like Beatles and Elvis and so forth, that were really
a function of distribution, I think, obviously of talent and innovation, but
there was also a technical component. And of course the next big innovation to
come along was radio, broadcast radio. And I think radio is interesting
because it started not as a music medium, it
started as an information medium for news and
then radio needed to find something to fill the time with so that they could
honestly play more ads and make more money, and music was
free. So then you had this massive distribution where you could program to
people. I think those things, that ecosystem,
is what created the ability for hits. But it was also a very broadcast
medium, so you would tend to get these massive,
massive hits, but maybe not such a long tail.
In terms of choice, of everybody listening to the same stuff.
Yeah, and as you said, I think there are some social benefits to that.
I think, for example, there's a high statistical chance that if I talk
about the latest episode of Game of Thrones, we have something to talk about
just statistically. In the age of individual choice, maybe some of that
goes away. So I do see the value of
shared cultural components, but I also obviously love personalization.
So let's catch this up to the internet. So
maybe Napster, well first of all, there's like mp3s,
there's like tape, CDs. There was a digitalization of music with a CD, really. It was
physical distribution, but the music became
digital. And so they were files, but basically boxed software,
to use a software analogy. And then you could start downloading these files.
And I think there are two interesting things that happened. Back to
music used to be longer before it was constrained by the distribution medium.
I don't think that was a coincidence. And then really the only music genre to have
developed mostly after music was a file again on the
internet is EDM. And EDM is often much longer than the
traditional music. I think it's interesting to think
about the fact that music is no longer constrained in
minutes per song or something. It's a legacy of an old distribution
technology. And you see some of this new music that
breaks the format. Not so much as I would have expected actually by now,
but it still happens. So first of all, I don't really know what EDM is.
Electronic dance music. You could say Avicii
was one of the biggest in this genre. So the main constraint is of time. Something
that has three, four, five minutes on. So you could have songs that were eight
minutes, ten minutes and so forth. Because it started as a digital
product that you downloaded. So you didn't have
this constraint anymore. So I think it's something really
interesting that I don't think has fully happened yet.
We're kind of jumping ahead a little bit to where we are. But I think there's
tons of formal innovation in music that should happen now. That couldn't
happen when you needed to really adhere to the distribution constraints.
If you didn't adhere to that, you would get no distribution.
So Björk for example, the Icelandic artist,
§02Transcript · 13:29–26:48
she made a full iPad app as an album. That was very expensive.
Even though the app still has great distribution,
she gets nowhere near the distribution versus staying within the three minute
format. So I think now that music is fully
digital inside these streaming services, there is
the opportunity to change the format again and allow creators to be much more
creative without limiting their distribution ability.
That's interesting that you're right. It's surprising that we don't see that
taking advantage more often. It's almost like the constraints of the
distribution from the 50s and 60s have molded the culture to where we
want the three to five minutes on than anything else.
So we want the song as consumers and as artists.
Because I write a lot of music and I never even thought about writing
something longer than 10 minutes.
It's really interesting that those constraints. Because all your
training data has been three and a half minute songs.
It's right. So yeah, digitization of data
led to then MP3s. Yeah, so I think you had this file then
that was distributed physically. But then you had the components of digital
distribution. And then the internet happened.
And there was this vacuum where you had a format that could be digitally shipped,
but there was no business model. And then all these pirate networks
happened. Napster and in Sweden, Pirate Bay,
which was one of the biggest. And I think from a consumer
point of view, which kind of leads up to the inception of
Spotify from a consumer point of view, consumers for the first time had this
access model to music where they could, without kind of any
marginal cost, they could try different tracks.
You could use music in new ways. There was no marginal cost.
And that was a fantastic consumer experience to have access to all the
music ever made. I think was fantastic. But it was also horrible for artists
because there was no business model around it. So they didn't make any money.
So the user need almost drove the user interface before there was a
business model. And then there were these download
stores that allowed you to download files, which was a solution, but it didn't
solve the access problem. There was still a marginal cost of 99
cents to try one more track. And I think that that heavily limits how
you listen to music. The example I always give is
in Spotify, a huge amount of people listen to music while they sleep, while
they go to sleep and while they sleep. If that costed you 99 cents per three
minutes, you probably wouldn't do that. And you would be much less
adventurous if there was a real dollar cost to exploring music.
So the access model is interesting in that it changes your music behavior.
You can be, you can take much more risk because there's no marginal cost to it.
Maybe let me linger on piracy for a second because I find,
especially coming from Russia, piracy is something that's very interesting.
To me,
not me, of course, ever, but I have friends who've partook in piracy
of music, software, TV shows, sporting events. And usually to me what
that shows is not that they can actually pay the
money and they're not trying to save money.
They're choosing the best experience. So what to me piracy shows is a business
opportunity in all these domains. And that's where I think you're right.
Spotify stepped in, is basically piracy was an experience. You can
explore, find music you like, and actually the interface of piracy is
horrible because it's, I mean, it's bad metadata.
Yeah, bad metadata, long download times, all kinds of stuff.
And what Spotify does is basically first rewards artists and second
makes the experience of exploring music much better. I mean, the same is true,
I think, for movies and so on. Piracy reveals,
in the software space, for example, I'm a huge user and fan of Adobe products
and there was much more incentive to pirate Adobe products
before they went to a monthly subscription plan.
And now all of the said friends that used to pirate Adobe products that I
know now actually pay gladly for the
monthly subscription. I think you're right. I think it's a sign
of an opportunity for product development
and that sometimes there's a product market fit
before there's a business model fit in product development. I think that's
a sign of it. In Sweden, I think it was a bit of both.
There was a culture where we even had a political party called
the Pirate Party and this was during the time when
people said that information should be free. It was somehow wrong to
charge for ones and zeros. So I think people
felt that artists should probably make money somehow else
and concerts or something. So at least in Sweden, it was part
really social acceptance, even at the political level.
But that also forced Spotify to compete with
with free, which I don't think would actually could have happened
anywhere else in the world. The music industry needed to be
doing bad enough to take that risk and Sweden was like the perfect testing
ground. It had government funded high bandwidth, low
latency broadband, which meant that the product would work
and it was also there was no music revenue anyway. So they were kind of
like, I don't think this is going to work but
why not? So this product is one that I don't think could have happened in
America, the world's largest music market, for example.
So how do you compete with free? Because that's an interesting world
of the internet where most people don't like to pay for things.
So Spotify steps in and tries to, yes, compete with free.
How do you do it? So I think two things. One is
people are starting to pay for things on the internet. I think
one way to think about it was that advertising was the first
business model because no one would put a credit card on internet. Transactional
with Amazon was the second and maybe subscription is the third
and if you look offline, subscription is the biggest of those.
So that may still happen. I think people are starting to pay but definitely back
then we needed to compete with free and the first thing you need to do is
obviously to lower the price to free and then you need to be better somehow
and the way that Spotify was better was on the user experience, on the
actual performance, the latency of, you know, even if you had
high bandwidth broadband, it would still take you 30 seconds to a minute to
download one of these tracks. So the Spotify experience of starting
within the perceptual limit of immediacy, about 250 milliseconds,
meant that the whole trick was it felt as if you had downloaded all of PirateBay.
It was on your hard drive. It was that fast even though it wasn't
and it was still free but somehow you were actually still
being a legal citizen. That was the trick that Spotify managed to
to pull off. So I've actually heard you
say this or write this and I was surprised that I wasn't aware of it
because I just took it for granted. You know, whenever an awesome thing
comes along you're just like, "Oh, of course it has to be this way.
That's exactly right." That it felt like the entire world's libraries at my
fingertips because of that latency being reduced.
What was the technical challenge in reducing the latency?
So there was a group of really, really talented engineers.
One of them called Ludwig Stregius. He wrote the...
actually from Gothenburg. He wrote the initial...
the uTorrent client, which is kind of an interesting backstory to Spotify.
You know, that we have one of the top developers from
BitTorrent clients as well. So he wrote uTorrent, the world's smallest
BitTorrent client. And then he was acquired very early by
Daniel and Martin, who founded Spotify. And they actually sold the uTorrent
client to BitTorrent but kept Ludwig. So Spotify had a lot of experience
within peer-to-peer networking. So the original
innovation was a distribution innovation, where Spotify built an
end-to-end media distribution system up until only a few years ago. We actually
hosted all the music ourselves. So we had both the server side and the
client and that meant that we could do things such as having
a peer-to-peer solution to use local caching
on the client side, because back then the world was mostly desktop.
But we could also do things like hack the TCP protocols,
things like Nagel's algorithm for kind of exponential back-off
or ramp up and just go full throttle and optimize for latency
at the cost of bandwidth. And all of this end-to-end control meant that we
could do an experience that felt like a step
change. These days we actually are on GCP. We don't host our own
stuff and everyone is really fast these days. So that was the initial
competitive advantage. But then obviously you have to move on over time.
And that was over 10 years ago, right? That was in 2008. The product
was launched in Sweden. It was in a beta, I think, 2007. And it was on the desktop,
right? So it was desktop only. There's no phone.
There was no phone. The iPhone came out in 2008,
but the App Store came out one year later, I think. So the writing was on the
wall, but there was no phone yet. You've mentioned that people would
use Spotify to discover the songs they like and then they would
torrent those songs so they can copy it to their phone.
Just hilarious. Exactly. Not torrent, pirate.
Seriously, piracy does seem to be like a good guide for business models.
Video content. As far as I know, Spotify doesn't have video content.
Well, we do have music videos and we do have videos on the
service, but the way we think about ourselves is that we're an audio
service and we think that if you look at the amount of
time that people spend on audio, it's actually very similar to the amount of
time that people spend on video. So the opportunity should be equally
big, but today it's not at all valued. Video is valued much higher. So we
think it's basically completely undervalued. We think of
ourselves as an audio service, but within that audio service, I think
video can make a lot of sense. I think for
when you're discovering an artist, you probably do want to see them
and understand who they are, to understand their identity.
You won't see that video every time. No, 90% of the time the phone is going to be
in your pocket. For podcasters, you use video. I think
that can make a ton of sense. So we do have video, but we're an audio
service where, think of it as we call it internally
backgroundable video. Video that is helpful, but isn't
the driver of the narrative. I think also if we look at
YouTube, the way people, there's quite a few folks who
listen to music on YouTube. So in some sense, YouTube is a bit of a competitor
to Spotify, which is very strange to me that people use YouTube to listen
to music. They play essentially the music videos,
right, but don't watch the videos and put it in their pocket.
Well, I think it's similar to what, strangely, maybe it's similar to
what we were for the piracy networks, where
YouTube, for historical reasons, have a lot of music videos.
So people use YouTube for a lot of the discovery part of the process, I
think. But then it's not a really good sort of
"MP3 player" because it doesn't even background. Then you have to keep
the app in the foreground. So it's not a good consumption tool,
but it's a decently good discovery tool. I mean, I think YouTube is a fantastic
product and I use it for all kinds of purposes.
That's true. If I were to admit something, I do use YouTube a little bit
for the discovery, to assist in the discovery process of songs.
And then if I like it, I'll add it to Spotify.
But that's OK. That's OK with us. OK, so sorry, we're jumping around a little bit.
So this kind of incredible, you look at
Napster, you look at the early days of Spotify.
How do you, one fascinating point is, how do you grow a user base?
So you're there in Sweden, you have an idea.
I saw the initial sketches that look terrible.
How do you grow a user base from a few folks to
millions? I think there are a bunch of tactical answers.
So first of all, I think you need a great product. I don't think you take a bad
product and market it to be successful.
So you need a great product. But sorry to interrupt, but it's a totally new way to
listen to music, too. So it's not just... Did people realize immediately that
Spotify is a great product? I think they did. So back to the point of
piracy, it was a totally new way to listen to music legally.
But people had been used to the access model in Sweden
and the rest of the world for a long time through piracy. So one way to think
§03Transcript · 26:51–39:57
about Spotify, it was just legal and fast piracy.
And so people have been using it for a long time. So they weren't alien to it.
They didn't really understand how it could be legal because it would seem too
fast and too good to be true. Which I think is a great product
proposition if you can be too good to be true.
But what I saw again and again was people showing each other, clicking the
song, showing how fast it started and saying, "Can you believe this?"
So I really think it was about speed. Then we also had an invite
program that was really meant for scaling because we hosted our own
servers. We needed to control scaling. But that built a lot of expectation and
I don't want to say hype because hype implies that it was
that it wasn't true. Excitement around the product. And we've
replicated that when we launched in the US.
We also built up an invite-only program first. So lots of tactics.
But I think you need a great product that solves some problem.
And basically the key innovation, there was technology, but on a metal
level, the innovation was really the access model versus the ownership model.
And that was tricky. A lot of people said that they
wanted to own their music. They would never kind of rent it or
borrow it. But I think the fact that we had a free
tier, which meant that you get to keep this music for life as well,
helped quite a lot. So this is an interesting psychological point
that maybe you can speak to. It was a big shift for me.
It's almost like I had to go to therapy for this.
I think I would describe my early listening experience, and I think a lot
of my friends do, is basically hoarding music. It's you're
like slowly, one song by one song or maybe albums, gathering a collection
of music that you love. And you own it. It's like often,
especially with CDs or tape, you like physically had it.
And what Spotify, what I had to come to grips with, it was kind of
liberating actually, is to throw away all the music.
I've had this therapy session with lots of people.
And I think the mental trick is, so actually we've seen the user data when
Spotify started, a lot of people did the exact same thing. They started hoarding
as if the music would disappear, right? Almost the equivalent of downloading.
And so, you know, we had these playlists that had limits of like
a few hundred thousand tracks, and we figured no one will ever. Well, they do.
Hundreds and hundreds and hundreds of thousands of tracks. And to this day,
you know, some people want to actually save, quote unquote, and play the entire
catalog. But I think that the therapy session goes
something like, instead of throwing away your music,
if you took your files and you stored them in a locker
at Google, it'd be a streaming service. It's just that in that locker, you have
all the world's music now for free. So instead of giving away your music, you
got all the music. It's yours. You could think of it as
having a copy of the world's catalog there forever. So you actually got
more music instead of less. It's just that you just took that hard
disk and you sent it to someone who stored it for you. And once
you go through that mental journey of like, still my files, they're just over
there, and I just have 40 million of them, 50
million of them or something now. Then people are like, okay, that's good.
The problem is, I think, because you paid us a subscription,
if we hadn't had the free tier where you would feel like, even if I don't want to
pay anymore, I still get to keep them. You keep your
playlist forever. They don't disappear even though you stop paying.
I think that was really important. If we would have started as,
you know, you can put in all this time, but if you stop paying, you lose all your
work. I think that would have been a big
challenge and was the big challenge for a lot of our competitors. That's another
reason why I think the free tier is really important. That people need to
feel the security that the work they put in, it will never disappear, even if they
decide not to pay. I like it how you put the work you put in.
I actually stopped even thinking of it that way. I just,
actually Spotify taught me to just enjoy music.
That's great. As opposed to what I was doing before, which is like
in an unhealthy way, hoarding music. Which I found that because I was doing
that, I was listening to a small selection of
songs way too much to where I was getting sick of them.
Whereas Spotify, the more liberating kind of approach is I was just enjoying.
Of course, I listened to "Stairway to Heaven" over and over, but
because of the extra variety, I don't get as sick of them.
There's an interesting statistic I saw. So,
Spotify has, maybe you can correct me, but over 50 million songs,
tracks and over 3 billion playlists. So, 50 million songs and 3 billion
playlists. 60 times more playlists. What do you make of that?
Yeah, so the way I think about it is that
from a statistician or machine learning point of view,
you have all these, if you want to think about reinforcement learning, you
have this state space of all the tracks and you can
take different journeys through this world.
I think of these as like people helping themselves and each other
creating interesting vectors through this space of tracks.
Then it's not so surprising that across many tens of millions of
atomic units, there will be billions of paths
that make sense. We're probably pretty quite far away from
having found all of them. So, kind of our job now
is users, when Spotify started, it was really
a search box that was for the time pretty powerful. Then
I like to refer to this programming language called playlisting,
where if you, as you probably were pretty good at music,
you knew your new releases, you knew your back catalog, you knew your "Starry Way
to Heaven", you could create a soundtrack for
yourself using this playlisting tool that's like meta programming language for
music to soundtrack your life. People who were
good at music, it's back to how do you scale the product.
For people who are good at music, that wasn't actually enough. If you had the
catalog and a good search tool, you can create your own sessions, you
could create really good a soundtrack for your entire life.
Probably perfectly personalized because you did it yourself.
But the problem was most people, many people aren't that good at music, they
just can't spend the time. Even if you're very good at music, it's
gonna be hard to to keep up. So what we did to try to scale this was to
essentially try to build, you can think of them as agents, that
this friend that some people had that helped them navigate this music
catalog, that's what we're trying to do for you.
But also there is something like 200 million active users on Spotify.
So there, okay, so from the machine learning perspective,
you have these 200 million people plus, they're creating, it's really
interesting to think of playlists as, I mean, I don't know if you meant it
that way, but it's almost like a programming language. It's
or at least a trace of exploration of those individual agents,
the listeners. And you have all this new tracks coming in. So it's a
fascinating space that is ripe for machine learning.
So is there, is it possible, how can playlists be used as data
in terms of machine learning and to help Spotify organize the music?
So we found in our data, not surprising, that people who playlisted a lot,
they retained much better, they had a great experience. And so our first
attempt was to playlist for users. And so we acquired
this company called Tunigo of editors and professional playlisters
and kind of leveraged the maximum of human intelligence
to help build kind of these vectors
through the track space for people. And that broadened the product.
Then the obvious next, and we used statistical means
where they could see when they created a playlist, how did that playlist
perform? They could see skips of the songs, they could see how the
songs perform, and they manually iterated the playlist to maximize
performance for a large group of people. But there
were never enough editors to playlist for you personally. So the promise of
machine learning was to go from kind of group personalization
using editors and tools and statistics to individualization. And then what's so
interesting about the three billion playlists we have is,
we ended, the truth is we lucked out. This was not a priority strategy, as is
often the case. It looks really smart in hindsight, but
it was dumb luck. We looked at these playlists and
we had some people in the company, a person named Eric Bernadson,
who was really good at machine learning already back then, in like 2007,
2008. Back then it was mostly collaborative
filtering and so forth. But we realized that what this is, is
people are grouping tracks for themselves that have some semantic
meaning to them. And then they actually label it with a
playlist name as well. So in a sense, people were grouping
tracks along semantic dimensions and labeling them.
And so could you use that information to find that
latent embedding? And so we started playing around with
collaborative filtering and we saw tremendous success with it.
Basically trying to extract some of these
dimensions. And if you think about it, it's not surprising at all.
It'd be quite surprising if playlists were actually random, if they had no
semantic meaning. For most people, they group these
tracks for some reason. So we just happened across this
incredible data set where people are taking
these tens of millions of tracks and grouped them along
different semantic vectors. And the semantics being outside the
individual users, so it's some kind of universal.
There's a universal embedding that holds across
people on this earth. Yes, I do think that
the embeddings you find are going to be reflective of the people who playlisted.
So if you have a lot of indie lovers who playlist,
your embed is going to perform better there. But what we found was that,
yes, there were these latent similarities.
They were very powerful. And we had, it was interesting because
I think that the people who playlisted the most initially
were the so-called music aficionados who were really into music. And they often
had a certain, their taste was often
geared towards a certain type of music. And so what surprised us, if you look at
the problem from the outside, you might expect that the algorithms
would start performing best with mainstreamers first because
it somehow feels like an easier problem to solve mainstream taste
than really particular taste. It was the complete opposite for us.
The recommendations performed fantastically for people who saw
themselves as having very unique taste. That's probably
because all of them playlisted and they didn't perform so well for
mainstreamers. They actually thought they were a bit too
particular and unorthodox. So we had the complete
opposite of what we expected. Success within the hardest problem first
and then had to try to scale to more mainstream recommendations.
So you've also acquired EchoNest that analyzes song data.
So in your view, maybe you can talk about, so what kind of data is there from a
machine learning perspective? There's a huge amount, we're
talking about playlisting and just user data of what people are
listening to, the playlist they're constructing
and so on. And then there's the actual data within a song.
What makes a song, I don't know, the actual
waveforms. How do you mix the two? How much value is there in each? To me
it seems like user data is a romantic notion that the song
itself would contain useful information. But if I were to guess,
user data would be much more powerful. Like playlists would be much more
powerful. Yeah, so we use both. Our biggest success
initially was with playlist data without understanding
anything about the structure of the song. But when we acquired EchoNest, they had
the inverse problem. They actually didn't have any
play data. They were just a provider of recommendations, but they
didn't actually have any play data. So they looked at the structure of
songs sonically and they looked at Wikipedia for
cultural references and so forth, right? And did a lot of NLU and so forth. So we
got that skill into the company and combined
kind of our user data with their
content-based. So you can think of it as we were user-based and they were
content-based in their recommendations. And we combined those two. And for some
cases where you have a new song that has no
§04Transcript · 39:59–52:59
play data, obviously you have to try to go by
either who the artist is or the sonic information in the song or what
it's similar to. So there's definitely value in both and
we do a lot in both. But I would say yes, the user data captures things that
have to do with culture in the greater society
that you would never see in the content itself.
But that said, we have seen, we have a research lab in
Paris when we can talk more about that on
kind of machine learning on the creator side. What it can do for creators, not
just for the consumers. But where we looked at how does the
structure of a song actually affect the listening behavior? And it turns out
that there is a lot of, we can predict things
like skips based on the song itself. We could
say that maybe you should move that chorus a bit
because your skip is going to go up here.
There is a lot of latent structure in the music, which is not surprising
because it is some sort of mind hack. So there should be structure. That's
probably what we respond to. You just blew my mind actually
from the creator perspective. So that's a really interesting topic
that probably most creators aren't taking advantage of.
So I've recently got to interact with a few
folks, YouTubers, who are like obsessed with this idea of
what do I do to make sure people keep watching
the video? And they like look at the analytics of which point do people turn
it off and so on. First of all, I don't think that's
healthy because you can do it a little too much.
But it is a really powerful tool for helping the creative process.
You just made me realize you could do the same thing for
creation of music. So is that something you've looked into?
Can you speak to how much opportunity there is for that?
Yeah, I listened to the podcast with Zoroash
and I thought it was fantastic and I reacted to the same thing where he said
he posted something in the morning,
immediately watched the feedback, where the drop-off was and then responded to
that in the afternoon. Which is quite different from how
people make podcasts for example. I mean the feedback loop is almost
non-existent. So if we back out one level, I think
actually both for music and podcasts, which we also
do at Spotify, I think there's a tremendous opportunity
just for the creation workflow. I think it's really interesting speaking
to you, because you're a musician, a developer
and a podcaster. If you think about those three
different roles, if you make the leap as a musician,
if you think about it as a software tool chain, really,
your DAW with the stems, that's the IDE, right? That's where you work in source
code format with what you're creating.
Then you sit around and you play with that and when you're happy you compile
that thing into some sort of AAC or MP3 or something.
You do that because you get distribution. There are so many run times for that MP3
across the world in car stares and stuff. So
you kind of compile this executable and you ship it out in kind of an old-fashioned
boxed software analogy. And then you hope for the
best, right? But as a software developer,
you would never do that. First you go on GitHub and you collaborate with other
creators. And then you think it'd be crazy to
just ship one version of your software without doing an A/B test,
without any feedback loop. Issue tracking.
Exactly. And then you would look at the feedback loops and try to optimize
that thing, right? So I think if you think of it as a very
specific software tool chain, it looks quite arcane.
The tools that a music creator has versus what a software developer has.
So that's kind of how we think about it. Why wouldn't a
music creator have something like GitHub where you could collaborate
much more easily? So we bought this company called Soundtrap,
which has a kind of Google Docs for music approach,
where you can collaborate with other people on the kind of source code format
with stems. And I think introducing things like
AI tools there to help you as you're creating music,
both in helping you put accompaniment to your music,
like drums or something, help you master and mix automatically,
help you understand how this track will perform. Exactly what you would expect
as a software developer. I think it makes a lot of sense. And I
think the same goes for a podcaster. I think podcasters will expect to
have the same kind of feedback loop that Zirosh has.
Like, why wouldn't you? Maybe it's not healthy, but...
Sorry, I wanted to criticize the fact that you can overdo it.
Because a lot of the... And we're in a new era
of that, so you can become addicted to it.
And therefore, what people say, you become a slave to the YouTube algorithm.
It's always a danger of a new technology,
as opposed to, say, if you're creating a song,
becoming too obsessed about the intro riff to the song that keeps people
listening, versus actually the entirety of the creation process.
It's a balance. But the fact that there's zero...
I mean, you're blowing my mind right now, because you're
completely right that there's no signal whatsoever,
there's no feedback whatsoever on the creation process in music or podcasting,
almost at all. And are you saying that Spotify is hoping to help create tools
to... Not tools, but... - No, tools, actually. - Actually tools for creators.
- Absolutely. So we have... We've made some acquisitions the last few years
around music creation. This company called Soundtrap, which is a
digital audio workstation, but that is browser-based.
And their focus was really the Google Docs approach, where you can collaborate
with people much more easily than you could in previous tools. So we
have some of these tools that we're working with that we want to make
accessible, and then we can connect it with our
consumption data. We can create this feedback loop where
we could help you understand, we could help you
create and help you understand how you will perform. We also
acquired this other company within podcasting called Anchor, which is one of
the biggest podcasting tools, mobile-focused, so really focused on
simple creation or easy access to creation. But that also
gives us this feedback loop. And even before that, we
invested in something called Spotify for Artists and Spotify for Podcasters,
which is an app that you can download, you can verify that you are that creator.
And then you get things that software developers have had for
years. You can see where, if you look at your podcast, for example,
on Spotify or a song that you release, you can see
how it's performing, which cities it's performing in, who's listening to it,
what's the demographic breakup. So similar in the sense that you can
understand how you're actually doing on the platform.
So we definitely want to build tools. I think you also interviewed the
head of research for Adobe, and I think that's an,
back to Photoshop that you like, I think that's an interesting analogy as
well. Photoshop, I think, has been very
innovative in helping photographers and artists, and I think
there should be the same kind of tools for
for music creators, where you could get AI assistance, for example, as you're
creating music, as you can do with Adobe, where you can,
I want a sky over here, and you can get help creating that sky.
The really fascinating thing is what Adobe
doesn't have is a distribution for the content you create.
So you don't have the data of, if I create,
if I, you know, whatever creation I make in Photoshop or Premiere,
I can't get like immediate feedback like I can on YouTube, for example, about
the way people are responding. And if Spotify is creating those tools,
that's a really exciting, actually, world.
But let's talk a little about podcasts. So I have trouble talking to one
person, so it's a bit terrifying and kind of
hard to fathom, but on average, 60 to 100,000
people will listen to this episode. Okay, so it's intimidating.
Yeah, it's intimidating. So I hosted on Blueberry.
I don't know if I'm pronouncing that correctly, actually. It looks like most
people listen to it on Apple Podcasts, Castbox, and Pocketcast, and only about
a thousand listen on Spotify. Just my podcast, right?
So where do you see a time when Spotify will
dominate this? So Spotify is relatively new into this.
In podcasting. Sorry, yeah, in podcasting.
What's the deal with podcasting and Spotify?
How serious is Spotify about podcasting? Do you see a time where everybody would
listen to, you know, probably a huge amount of people,
majority perhaps, listen to music on Spotify? Do you see a
time when the same is true for podcasting? Well, I certainly hope so.
That is our mission. Our mission as a company is actually to
enable a million creators to live off of their art and a billion people be
inspired by it. And what I think is interesting about that mission is
it actually puts the creators first, even though it started as a consumer-focused
company, and it says to be able to live off of
their art, not just make some money off of their art as well.
So it's quite an ambitious project. And
so we think about creators of all kinds and
we kind of expanded our mission from being music to being
audio a while back. And that's not so much because
we think we made that decision. We think that decision was
was made for us. We think the world made that decision. Whether we like it or not,
when you put in your headphones, you're going to make a choice between
music and a new episode of your podcast or something else.
We're in that world whether we like it or not. And that's how radio works.
So we decided that we think it's about audio.
You can see the rise of audiobooks and so forth. We think audio is this great
opportunity. So we decided to enter it. And obviously
Apple and Apple Podcasts is absolutely dominating
in podcasting. And we didn't have a single podcast
only like two years ago. What we did though was
we looked at this and said, "Can we bring something to this?"
We want to do this, but back to the original Spotify, we had to do
something that consumers actually value to be able to do this. And the reason
we've gone from not existing at all to being the
quite a wide margin, the second largest podcast
consumption, still wide gap to iTunes, but we're growing quite fast.
I think it's because when we looked at the consumer problem,
people said surprisingly that they wanted their podcasts and
music in the same application. So what we did was we took a little
bit of a different approach where we said instead of building a separate
podcast app, we thought, "Is there a consumer problem to solve
here because the others are very successful already?"
And we thought there was in making a more seamless experience
where you can have your podcast and your music in the same application.
Because we think it's audio to you and that has been successful and
that meant that we actually had 200 million people to
offer this to instead of starting from zero.
So I think we have a good chance because we're taking a different approach than
the competition. And back to the other thing I mentioned
about creators, because we're looking at the
end-to-end flow, I think there's a tremendous amount of
innovation to do around podcasts as a format.
When we have creation tools and consumption, I think we could
start improving what podcasting is. I mean podcast is this
this opaque big like one two hour file that you're streaming, which it really
doesn't make that much sense in 2019 that
it's not interactive, there's no feedback loops, nothing like that.
So I think if we're gonna win it's gonna have to be because we build a better
product for creators and for consumers. So we'll
see, but it's certainly our goal. We have a long way to go.
Well the creators part is really exciting. You already got me
hooked there. It's the only stats I have. Blueberry just recently added the stats
of whether it's listened to the end or not.
And that's like a huge improvement, but that's still
nowhere to where you could possibly go in terms of statistics. You just download
the Spotify podcasters app and verify and then
§05Transcript · 53:01–1:05:33
then you'll know where people dropped out in this episode. Oh wow, okay.
The moment I started talking, okay. I might be depressed by this.
But okay, so one other question. The original Spotify for music,
and I have a question about podcasting in this line, is
the idea of albums. I have music aficionados, friends who are
really big fans of music, often really enjoy
albums, listening to entire albums of an artist.
Correct me if I'm wrong, but I feel like Spotify has helped
replace the idea of an album with playlists.
So you create your own albums. It's kind of the way, at least I've
experienced music and I really enjoy it that way.
One of the things that was missing in podcasting for me,
I don't know if it's missing. I don't know. It's an open question for me.
But the way I listen to podcasts is the way I would listen to albums.
So I take Joe Rogan Experience, and that's an album.
And I listen, you know, I put that on, and I listen one episode after the next,
then there's a sequence and so on. Is there room for
doing what you did for music, doing what Spotify did for music,
but creating playlists, sort of this kind of playlisting idea of
breaking apart from podcasting, from individual podcasts and creating
kind of this interplay? Or have you thought about
that space? It's a great question. So I think in
music, you're right. Basically, you bought an album. So it was like you bought
a small catalog of like 10 tracks, right? It was, again, it was actually a
lot of consumption. You think it's about what you like,
but it's based on the business model. Right. So you paid for this 10-track
service, and then you listen to that for a while. And then when everything was
flat-priced, you tended to listen differently.
Now, so I think the album is still tremendously important. That's
why we have it. And you can save albums and so forth. And
you have a huge amount of people who really listen according to albums.
And I like that because it is a creator format. You can tell a longer story
over several tracks. And so some people listen to just one track. Some people
actually want to hear that whole story. Now, in podcast, I think
it's different. You can argue that podcasts might be more like shows on
Netflix. You have like a full season of Narcos,
and you're probably not going to do like one episode of Narcos and then one of
House of Cards. There's a narrative there, and you
love the cast and you love these characters. So I think people will
love shows, and I think they will
listen to those shows. I do think you follow a bunch of shows at the same
time. So there's certainly an opportunity to bring you the latest episode of
whatever the five, six, ten things that you're into.
But I think people are going to listen to
specific hosts and love those hosts for a long time because I think there's
something different with podcasts where this
format of the experience of the
audience is actually sitting here right between us.
Whereas if you look at something on TV, the audio actually would come from,
you would sit over there, and the audio would come to you from both of us as if
you were watching, not as you were part of the conversation.
So my experience is having listened to podcasts like yours
and Joe Rogan, I feel like I know all of these people. They have no idea
who I am, but I feel like I've listened to so many hours of them.
It's very different from me watching a TV show or an interview.
So I think you kind of fall in love with people
and experience it in a different way. So I think
shows and hosts are going to be very important. I don't think
that's going to go away into some sort of thing where
you don't even know who you're listening to. I don't think that's going
to happen. What I do think is, I think there's a
tremendous discovery opportunity in podcasts
because the catalog is growing quite quickly.
And I think podcasts is only a few, like five, six hundred thousand shows
right now. If you look back to YouTube, that's
another analogy of creators. No one really knows if you would lift
the lid on YouTube, but it's probably billions
of episodes. And so I think the podcast catalog will probably grow
tremendously because the creation tools are getting easier.
And then you're going to have this discovery opportunity that I think is
really big. So a lot of people tell me that they love their shows,
but discovery in podcasts kind of suck. It's really hard to get into a new show.
They're usually quite long. It's a big time investment. So I think there's
plenty of opportunity in the discovery part.
Yeah, for sure. A hundred percent. And even the dumbest,
there's so many low-hanging fruit, too. For example,
just knowing what episode to listen to first
to try out a podcast. Exactly. Because most podcasts don't have an order to
them. They can be listened to out of order.
And sorry to say, some are better than others episodes.
So some episodes of Joe Rogan are better than others. And it's
nice to know which you should listen to to try it out.
And there's, as far as I know, almost no information
in terms of like upvotes on how good an episode is.
Exactly. So I think part of the problem is
it's kind of like music. There isn't one answer. People use music for different
things. And there's actually many different types of music. There's workout
music and there's classical piano music and focus music and
and so forth. I think the same with podcasts. Some podcasts are sequential.
They're supposed to be listened to in order. It's actually
telling a narrative. Some podcasts are one topic, kind of like
yours, but different guests. So you could jump in anywhere.
Some podcasts actually have completely different topics. And for those podcasts,
it might be that we should recommend one episode
because it's about AI from someone. But then they talk about
something that you're not interested in the rest of the episodes.
So I think what we're spending a lot of time on now is just first
understanding the domain and creating kind of the knowledge graph
of how do these objects relate and how do people consume. And I think we'll find
that it's going to be different. I'm excited.
Spotify is the first people I'm aware of that are
trying to do this for podcasting. Podcasting has been like a wild west
up until now. It's been a very... We want to be very careful though because it's
been a very good wild west. I think it's this fragile
ecosystem and we want to make sure that you
don't barge in and say like, "Oh, we're gonna
internetize this thing." And you have to think about the
creators. You have to understand how they get distribution today, who
listens to how they make money today, try to make sure that their
business model works, that they understand.
I think it's back to doing something, improving their products
like feedback loops and distribution. So jumping back into terms of this
fascinating world of recommender system and listening to music and using
machine learning to analyze things, do you think it's
better to... What currently, correct me if I'm wrong,
but currently Spotify lets people pick what they listen to for the
most part. There's a discovery process but you kind of
organize playlists. Is it better to let people pick what they listen to
or recommend what they should listen to? Something like Stations by Spotify that
I saw that you're playing around with. Maybe you can tell me what's the status
of that. This is a Pandora style app that just kind of...
As opposed to you select the music you listen to, it kind of
feeds you the music you listen to. What's the status of Stations by Spotify?
What's its future? The story of Spotify as we have grown
has been that we made it more accessible
to different audiences. Stations is another one of those where
the question is, some people want to be very specific. They actually want to hear
"Stairway to Heaven" right now. That needs to be very easy to do.
Some people or even the same person at some point might say
"I want to feel upbeat" or "I want to feel happy" or
"I want songs to sing in the car". So they put in
the information at a very different level and then we need to
translate that into what that means musically. So Stations is a test to
create like a consumption input vector that is much simpler where you can just
tune it a little bit and see if that increases the overall
reach. But we're trying to kind of serve the entire gamut of super advanced so-called
music aficionados all the way to people who
they love listening to music but it's not their number one priority in life.
They're not going to sit and follow every new release from every new
artist. They need to be able to influence music
at a different level. So we're trying, you
can think of it as different products and I think when
one of the interesting things to answer your question on
if it's better to let the user choose or to play, I think the answer is
the challenge when machine learning kind of came along
there was a lot of thinking about what does product development mean
in a machine learning context. People like Andrew Ng for example
when he went to Baidu he started doing a lot of practical machine learning, went
from academia and he thought a lot about this and he
had this notion that a product manager, designer, an engineer, they used to
work around this wireframe. Kind of describe what the product should look
like or something to talk about. When you're doing like a chatbot or a
playlist, what are you going to say? Like it should be good.
That's not a good product description. So how do you do that and he came up
with this notion that the test set is the new wireframe. The
job of the product manager is to source a good test set that is
representative of what, like if you say like I want to play this
that is Songstressing in the car. The job of the product manager is to go
and source like a good test set of what that means.
Then you can work with engineering to have algorithms to try to produce that
right. So we try to think a lot about how to
structure product development for a machine learning age and what we
discovered was that a lot of it is actually in the expectation
and you can go two ways.
Let's say that if you set the expectation with the user that this
is a discovery product like Discover Weekly,
you're actually setting the expectation that most of what we show you will not
be relevant. When you're in the discovery process
you're going to accept that actually if you find one gem every
Monday that you totally love, you're probably going to be happy.
Even though the statistical meaning one out of ten is terrible or one out of 20
is terrible from a user point of view because the setting was discovered is
fine. Can I say to interrupt real quick, I just
actually learned about Discover Weekly which is a Spotify,
I don't know, it's a feature of Spotify that shows you
cool songs to listen to. Maybe I can do issue tracking, I couldn't
find it on my Spotify app. It's in your library. It's in the library,
it's in the list of libraries because I was like whoa this is cool I
didn't know this existed and I tried to find it.
I will show it to you and feedback to our product team.
Yeah there you go but yeah so yeah sorry
just to mention the expectation there is
basically that you're going to discover new songs.
Yeah so then you can be quite adventurous in
the recommendations you do but we have another product called
Daily Mix which kind of implies that these are only going to be your
favorites. So if you have one out of ten that is
good and nine out of ten that doesn't work for you,
you're going to think it's a horrible product. So actually a lot of the product
development we learned over the years is about
setting the right expectations. So for Daily Mix, you know algorithmically
we would pick among things that feel very safe in
your taste space. With Discover Weekly we go kind of wild
because the expectation is most of this is not gonna. So a lot of
that, a lot of to answer your question there
a lot of should you let the user pick or not it depends.
We have some products where the whole point is that the user can click play
put the phone in the pocket and it should be really good music for like
an hour. We have other products where you probably need to say like no
no save no no and it's very interactive.
§06Transcript · 1:05:37–1:18:35
I see that makes sense and then the radio product the station's product is
one of these like click play put in your pocket for hours.
That's really interesting so you're thinking of different test sets
for different users and trying to create products that sort of optimize
optimize for those test sets that represent a specific set of users.
Yes I think one thing that I think is interesting is
we invested quite heavily in editorial in people creating playlists
using statistical data and that was successful for us and then we also
invested in machine learning and for the longest time you know within
Spotify and within the rest of the industry there was always this
narrative of humans versus the machine. Algo versus editorial and editors
would say like well if I had that data if I could see your
playlisting history and I made a choice for you I would have
made a better choice and they would have because they
understand they're much smarter than these algorithms. The human is
incredibly smart compared to our algorithms. They can take culture
into account and so forth. The problem is that they can't make 200
million decisions you know per hour for every user that
logs in so the algo may be not as sophisticated but much more
efficient. So there was this there was this
contradiction but then a few years ago we started
focusing on this kind of human in the loop thinking around machine learning
and we actually coined an internal term for it called algotorial
the combination of algorithms and editors where
if we take a concrete example you think of the editor
this paid expert that we have that's really good at something like
soul, hip-hop, EDM something right there are two experts no one in the industry
so they have all the cultural knowledge you think of them as the product manager
and you say that let's say that you want to create a
you think that there's a there's a product need in the world for something
like songs to sing in the car or songs to sing in the shower
I'm taking that example because it exists people love to scream
songs in the car when they drive right yeah so you want to create that product
then you have this product manager who's a musical expert
they create they come up with a concept like I think this is a missing thing in
humanity like a playlist called songs in the car
they create the the framing the image the title
and they create a test set of they create a group of songs like a few
thousand songs out of the catalog that they manually
curate that are known songs that are great to sing in the car
and they can take like true romance into account they understand things that our
algorithms do not at all so they have this huge set of tracks
then when we deliver that to you we look at your taste vectors and you
get the 20 tracks that are songs to sing in the car in your taste
so you have you have personalization and
editorial input in the same process if that makes sense yeah it makes
total sense and I have several questions around that this is a this is like
fascinating okay so first it is a little bit surprising to me
that the world expert humans are outperforming machines
at specifying songs to sing in the car so maybe you could talk to that a
little bit I don't know if you can put it into words but
what is it how difficult is this problem
uh of do you really uh I guess what I'm trying to ask is there
how difficult is it to encode the cultural references
uh the the context of the song the artists
all all those things together can machine learning really not do that
I mean I think machine learning is great at replicating patterns
if you have the patterns but if you try to write with me a spec of what songs
greatest song to sing in the car definition is is it is it loud does it
have many choruses should it have been in movies it's
it quickly gets incredibly complicated right yeah
and and a lot of it may not be in the structure of the song or the title it
could be cultural references because you know it was a history so so the
definition problems quickly get and I think that was the that
was the insight of Andrew Ng when he said the job of the product
manager is to understand these things that
that algorithms don't and then define what that looks like and then you have
something to train towards right then you have kind of the test set
and then so so today the editors create this pool of tracks and then we
personalize you could easily imagine that once you have this set you could
have some automatic exploration of the rest of the catalog
because then you understand what it is and then the other side of it when
machine learning does help is this taste vector how hard is it to
construct a vector that represents the things an
individual human likes this human preference so you can
you know music isn't like it's not like amazon
like things you usually buy music seems more amorphous like it's this
thing that's hard to specify like what what is well you know if you look at my
playlist what is the music that I love it's harder
it seems to be uh much more difficult to specify concretely
so how hard is it to build a taste vector
it is very hard in the sense that you need a lot of data
and I think what we found was that so it's not
so it's not a stationary problem it changes over time
um and so we've gone through the journey of if if um
you've done a lot of computer vision obviously I've done a bunch of computer
vision in my past and we started kind of with the
handcrafted heuristics for you know this is kind of in the music
this is this and if you consume this you probably like this
so we we have we started there and we have some of that still
then what was interesting about the playlist data was that you could find
these latent things that wouldn't necessarily even make sense to
you that could could even capture maybe
cultural references because they co-occurred
things that that wouldn't have appeared kind of mechanistically either in the
content or so forth so um
I think that um
I think the core assumption is that there are patterns
in in almost everything and if there are patterns
these these embedding techniques are getting better and better now now
as everyone else we're also using kind of deep embeddings where you can
encode binary values and and so forth um and and what I think is
interesting is is this process to try to find things
that um that do not necessarily you wouldn't
actually have have guessed so it is very hard in a in a in an
engineering sense to find the right dimensions it's an
incredible scalability problem to do for hundreds of millions of users and to
update it every day but in but in theory um
in theory embeddings isn't that complicated
the fact that you try to find some principal components or something like
that dimensionality reduction and so forth so
the theory I guess is easy the practice is
is very very hard and it's a it's a huge engineering challenge but fortunately we
have some amazing both research and engineering teams in
in this space yeah I guess the the question is all
I mean it's similar I deal with it with an autonomous vehicle space is the
question is how hard is driving and here is
basically the question is of edge cases uh so embedding probably works
not probably but I would imagine works well in a lot of cases
so there's a bunch of questions that arise then so do
song preferences does your taste vector depend on
context like mood right so there's different moods and
absolutely so how does that take in it is it is it possible to take that as a
consideration or do you just leave that as a interface
problem that allows the user to just control it
so when I'm looking for a workout music I kind of specify it by
choosing certain playlists doing certain search yeah
so that's a great point it's back to the product development
you could try to spend a few years trying to predict which mood you're in
automatically when you open Spotify or you create a tab which is happy and
sad right and you're going to be right 100% of the time with one click
now it's probably much better to let the user tell you if they're happy or sad
or if they want to work out on the other hand if your user interface become 2000
tabs you're introducing so much friction so
no one will use the product so then you have to get better
so it's this thing where I think maybe it was
I remember who coined it but it's called fault tolerant uis right you build a ui
that is tolerant to being wrong and then you can be much less right in
your in your in your algorithms so we you know
we've had to learn a lot of that building the right ui that
fits where the where the machine learning is
and and and a great discovery there which is which was by the teams during
uh one of our hack days was this thing of taking discovery packaging it
into a playlist and saying that these are new tracks
that we think you might like based on this and setting the right expectation
made it made it a great product so I think we
have this benefit that for example Tesla doesn't have that we can we can
we can change the expectation we can we can build a fault tolerant
setting it's very hard to be fault tolerant when you're driving at a
you know 100 miles per hour or something and and we we have the luxury of
being able to say that of being wrong if we have the right
ui which gives us different abilities to take more risk so I actually think
the self-driving problem is is much harder oh yeah
for sure it's much less fun because people die exactly
and since Spotify uh it's such a more fun problem because
failure will I mean failure is beautiful in a way it leads to exploration so it's
it's a really fun reinforcement learning problem the worst case scenario is you
get these wtf tweets like how the hell did I get this this song
which is which is a lot better than the self-driving failure
so what's the feedback that a user what's the signal
that a user provides into the system so the the you mentioned skipping
what is like the strongest signal is uh you didn't mention clicking like
so so we have a few signals that are important obviously
playing playing through so so one of the benefits of music actually even compared
to podcast or or movies is the object itself is really
only about three minutes so you get a lot of chances to recommend
and the feedback loop is is every three minutes instead of every
two hours or something so you actually get
kind of noisy but but quite fast feedback
and so you can see if people played through or if the which is you know the
inverse of skip really that's an important signal on the other
hand much of the consumption happens when your phone is in your pocket maybe
you're running or driving or you're playing on a speaker
and so you not skipping doesn't mean that you love that song it might be that
it wasn't bad enough that you would walk up and skip so it's a noisy signal
then then we have the equivalent of the like which is you saved it to your
library that's a pretty strong signal of
affection and then we have the more explicit signal of
playlisting like you took the time to create a playlist you put it in there
there's a very little small chance that if you took
all that trouble this is not a really important track to you
and then we understand also what other tracks it relates to so we have
we have the playlisting we have the like and then we have the listening or skip
and and you have to have very different approaches to all of them because at
different levels of of noise one one is very voluminous but
noisy and the other is rare but you can you can probably trust it yeah
it's interesting because uh i i think between those signals captures
all the information you'd want to capture i mean there's a feeling
a shallow feeling for me that there's sometimes i'll hear a song that's like
yes this is you know this is the right song for
the moment but there's really no way to express
that fact except by listening through it all the way
yeah and maybe playing it again at that time or something yeah
there's no need for a button that says this was the best song could have heard
at this moment well we're playing around with that with
kind of the thumbs up concept saying like i really like this
just kind of talking to the algorithm it's unclear if that's
the best way for humans to interact maybe it is maybe they should think of
spotify as a person an agent sitting there trying to serve you and you can
say like bad spotify good spotify right now the
§07Transcript · 1:18:39–1:31:14
analogy we've had is more you shouldn't think of of us we should
be invisible and the feedback is if you save it
kind of you work for yourself you do a playlist because you think is great and
we can learn from that it's kind of back to back to tesla how
they kind of have this shadow mode they sit in what you drive
we kind of took the same analogy we sit in what you playlist
and then maybe we can we can offer you an autopilot where you can take over for
a while or something like that and then back off if you say like that's
not that's not good enough but but i think it's interesting to figure
out what your mental model is if spotify is an ai that you talk to
which i think might be a bit too abstract for for many
consumers or if you still think of it as it's my music app
but it's just more helpful and depends on the device it's
running on which brings us to smart speakers
so i have a lot of the spotify listening i do is on
things that on devices i can talk to whether it's from amazon google or
apple what's the role of spotify on those
devices how do you think of it differently than
on the phone or on the desktop there are a few things to say about the
first of all it's incredibly exciting they're growing like
crazy especially here in the in the in the u.s
and it's solving a consumer need that i think is
is you can think of it as
just remote interactivity you can control this thing from from from across
the room and it may feel like a small thing but
it turns out that friction matters to consumers being
able to say play pause and so forth from across
the room is is very powerful so basically you made you made the
living room interactive now and
what we see in our data is that the number one use case for these speakers
is music music and podcast so fortunately for us it's been important
to these companies to have those use case covered so they
want to spotify on this we have very good relationships with
with them and we're seeing we're seeing tremendous
success with them what what i think it's interesting about them is
it's already working we we we kind of had this epiphany
many years ago back when we started using sonos if you went through all the
trouble of setting up your sonos system you had this magical experience where
you had all the music ever made in your living room and and we we we
made this assumption that the the home everyone used to have a cd
player at home but they never managed to get their files
working in the home having this network attached storage was too cumbersome for
most consumers so we made the assumption that the home
would skip from the cd all the way to the streaming box
where where you would get you would buy the stereo and have all the music built
in that took longer than we thought but with the voice speakers that was the
unlocking that made kind of the connected speaker
happen in the home so so it really it really exploded and
we saw this engagement that we predicted would happen
what i think is interesting though is where it's going from now
right now you think of them as voice speakers but i think if you look at
uh google io for example they just added a camera
to it where you know when the alarm goes off instead of saying
hey google stop you can just wave your hand
so i think they're going to think more of it as a
as an agent or as a as an assistant truly an assistant and an assistant that
can see you it's going to be much more effective than
than a blind assistant so i think these things will morph and we won't
necessarily think of them as quote-unquote voice speakers anymore
just as
interactive access to the internet in the home
but i still think that the biggest use case for those will be
will be audio so for that reason we're investing heavily in it
and we built our own nlu stack to be able to the the challenge here is
how do you innovate in that world it's it's it lowers friction for consumers
but it's also much more constrained there you have no pixels to play with
in an audio only world it's really the
vocabulary that is the interface so we started
investing and playing around quite a lot with that trying to understand
what the future will be of you speaking and gesturing and
waving at your music and actually uh you're actually nudging
closer to the autonomous vehicle space because from everything i've seen the
level of frustration people experience upon failure
of natural language understanding is much higher
than failure in other contexts people get frustrated really fast
so if you screw that experience up even just a little bit they give up really
quickly yeah and i think you see that in the data
while while it's tremendously successful the most common interactions are play
pause and you know next the things where if
you compare it to taking up your phone unlocking it bringing up the app and
skipping clicking skip yeah it was it was much
lower friction but then uh for for longer more
complicated things like can you find me that song
people still bring up their phone and search and then play it on their speaker
so we tried again to build a fault tolerant ui where for the more for the
more complicated things you can still pick up your phone have
powerful full keyboard search and then try to optimize for where there
is actually lower friction and try to it's it's kind of like the
test autopilot thing you have to be at the level where
you're helpful if you're too smart and just in the way people are going to get
frustrated and first of all i'm not obsessed with
stairway to heaven it's just a good song but let me mention that as a use case
because it's an interesting one i've literally told
one of i don't want to say the name of the speaker because it'll when people
are listening to it it'll make their speaker go off but i talk to the
speaker and i say play stairway to heaven and every time
it like not every time but a large percentage of the time plays the wrong
stairway to heaven it plays like some cover of the and
that part of the experience i actually wonder from a business perspective does
spotify control that entire experience or no
it seems like the nlu the the natural language stuff
is controlled by the speaker and then spotify stays at a layer below that
it's a good and complicated question some of which is
dependent on the on the partner so it's hard to comment on the on the specifics
but the question is the right one the
challenge is if you can't use any other
personalization i mean we know which stairway to heaven
and and the truth is maybe for for one person it is exactly the cover that they
want and they would be very frustrated if it
plays i i think we i think we default to the right version but
but you actually want to be able to do the cover for the person that just play
the cover 50 times or spotify is just going to seem stupid
so you want to be able to leverage the personalization but you have this stack
where where you have the the asr and this thing called the end best list of
the end best guesses here and then the person comes in at the
end you actually want the personalization to be here when you're
guessing about what they actually meant so we're working with these partners um
and it's a complicated it's a complicated thing where
you want to you want to be able so first of all you want to be very careful with
your users data you don't want to share your users data without their permission
but you want to share some data so that their experience gets better
um so that these partners can understand enough but not too much and so forth
so it's really the the trick is that it's like a business
driven relationship where you're doing product development across companies
together yeah which is which is really
complicated but this is exactly why we built our own
nlu so that we actually can make personalized guesses because this is the
biggest frustration from a user point of view they don't
understand about asrs and nbest lists and
and business deals they're like how hard can it be i've told this thing
50 times this version and still it plays the wrong thing it can't it can't be
hard so we try to take that user approach if
the user the user is not going to understand the
complications of business we have to solve it let's talk
about sort of a complicated subject that i myself i'm quite
torn about the idea sort of of um
paying artists right i saw as of august 31st
2018 over 11 billion dollars were paid to rights holders
so and further distributed to artists from spotify
so a lot of money is being paid to artists first of all
the whole time as a consumer for me when i look at spotify
i'm not sure i'm remembering correctly but i think you said exactly how i feel
which is this is too good to be true like
when i started using spotify i assumed you guys would go bankrupt in like a
month it's like this is too good a lot of
people did
it's like this is amazing uh so one question i have is sort of the
bigger question how do you make money in this complicated world
how do you deal with the relationship with record labels who
are complicated uh these big you're essentially in have the task
of herding cats but like rich and powerful cats
and also have the task of paying artists enough and paying
those labels enough and still making money in the internet space where people
are not willing to pay hundreds of dollars a month so how do
you navigate the space how do you navigate that's a beautiful
description herding rich cats yeah i've never heard that before
now it is very complicated and i think uh
certainly actually betting against spotify has been statistically a very
smart thing to do just looking at the at the line of roadkill in music
streaming services um it's it's kind of i think if i had
understood the complexity when i joined spotify
unfortunately fortunately i didn't know enough about
the the music industry to understand the complexities because then i would have
made a more rational guess that it wouldn't work
so you know ignorance is bliss but i think
there have been a few distinct challenges i think as i said one of the
things that made it work at all was that sweden and the nordics
was a lost market so um there were you know there was there was no risk
for labels to try this i don't think it would have worked if
if the market was uh was healthy so so that was the initial condition then
then we had this tremendous challenge with the model itself so
now most people were pirating but for the people who bought a download or a cd
the artists would get all the revenue for all the future plays
then right so you got it all up front whereas the streaming model was like
almost nothing day one almost nothing day two
and then at some point this curve of incremental revenue
would intersect with your day one payment and that took a long time to
play out before before um the music labels they understood
that but on the artist side it took a lot of time to understand that actually
if i have a big hit that is going to be played for for for many years this is a
much better model because i get paid based on how much
people use the product not how much they thought they would use
it day one or so forth so it was a complicated model to get
across and but time helped with that right and
now now the revenues to the music industry
actually are bigger again then you know it's gone through this
incredible dip and now they're back up and so we're
very we say proud of having having been a
part of that um so there have been distinct problems
i think when it comes to the to the labels
we have taken the painful approach some of our competition at the time they kind
of they kind of looked at other
companies and said if we just if we just ignore the rights
we get really big really fast we're going to be too big for the
for the labels to kind of too big to fail they're not going to kill us we
didn't take that approach we went legal from day one
and we we negotiated and negotiated and negotiated it was very slow it's very
frustrating we were angry at seeing other companies
taking shortcuts and seeming to get away with it
it was this this this game theory thing where over many rounds of playing the
§08Transcript · 1:31:18–1:44:09
game this would be the right strategy and even
though clearly there's a lot of frustrations
at times during renegotiations there is this there is this weird trust
where we have been honest and fair we've never screwed them they've never
screwed us it's tenuous but there's this trust and like they know
that if music doesn't get really big if lots
of people do not want to listen to music and want to pay for it
spotify has no business model so we actually are incredibly
aligned right other companies not to be tennis but other companies have other
business models where even if they made no music from no money for music
they'd still be profitable companies but spotify won't so and i think the
industry sees that we are actually aligned business-wise
so there is this this trust that allows us to
to do product development even if it's scary
um you know taking risks the free model itself
was an incredible risk for the music industry to take that they should get
credit for now some of it was that they had nothing to lose in sweden but
frankly a lot of the labels also took risk and so i
think we built up that trust with it with the i think uh hurting
with cats sounds a bit what's the word it sounds
like yeah dismissive of the cats dismissive
no every cat mattered they're all beautiful and very important
exactly they've taken a lot of risks and certainly it's been frustrating a lot of
good yeah so it's it's it's really like
playing it's it's game theory if you play the
if you play the game many times then you can have the statistical outcome that
you bet on and it feels very painful when you're in the middle of that
thing i mean there's risk there's trust there's relationships
from uh just having read the biography of steve jobs
similar kind of relationships were discussed in itunes
the idea of selling a song for a dollar was very uncomfortable
for labels and exactly and there was no it was the same kind of thing it was
trust it was game theory as as a lot of
relationships that had to be built and uh it's really a terrifyingly
difficult process that apple could go through a
little bit because they could afford for that process to fail for
spotify it seems terrifying because uh you can't initially i think a lot of it
comes out comes down to you know honestly daniel and his tenacity
in in negotiating which seems like an impossible
it's a fun task because you know he was completely unknown and so forth but
maybe that was also the reason that that it worked
but i think uh
yeah i think game theory is probably the best way to think about it you could
straight go straight for this like nash equilibrium that
someone is going to defect or or you play many times you try to actually
go for the top left the corporations sell is there any magical reason why
spotify seems to have won this so a lot of people have tried to do
what spotify tried to do and spotify has come out well so the
answer is that there's no magical reason because i don't believe in magic
but i think there are there are reasons um
and i think some of them are that people have
misunderstood a lot of what we actually do
the actual the actual spotify model is very complicated they've looked at the
premium model and said it seems like you can you can
charge 9.99 for music and people are going to pay but that's
not what happened actually when we launched the original mobile product
everyone said they would never pay what happened was they started on the on
the free product and then their engagement grew so much
that eventually they said maybe it is worth 9.99 right
it's uh it's your propensity to pay grows with your engagement
so we have this super complicated business model where you operate two
different business model advertising and premium at the same time
and i think that is hard to replicate i have i struggle to think of other
companies that run large-scale advertising and
subscription products at the same time so i think the business model is
actually much more complicated than people
think it is and and so some people went after just the premium part without the
free part and ran into a wall where no one wanted
to pay some people went after just music
music should be free just ads which doesn't give you enough revenue and
doesn't work for the music industry so i think that combination is um it's
kind of opaque from the outside so maybe i shouldn't say it here and
reveal the secret but that that turns out to be harder to
replicate than you would think so there's a lot of
brilliant business strategy here brilliance or luck probably more luck
but it doesn't really matter it looks brilliant in retrospect
let's call it brilliant yeah when the books are written it'll be brilliant
you've uh mentioned that your philosophy is to embrace change
so how will the music streaming and music listening world change over the
next 10 years 20 years you look out into the
far future what do you think i think that music and
for that matter audio podcasts audio books i think it's
one of the few core human needs i think it there is no good reason to
me why it shouldn't be at the scale of something like
messaging or social networking i don't think it's a niche thing
to listen to music or news or something so i think scale is obviously one of the
things that i really hope for i think i hope that it's going to be billions of
users i hope eventually everyone in the world gets access to all
the world's music ever made so obviously i think it's going to be a
much bigger business otherwise we we wouldn't be betting this big
uh now if you if you look more at how it is consumed what i'm hoping is back
to this analogy of the software tool chain
where i think i sometimes uh internally i make this analogy to
to text messaging text messaging was also based on
standards in the in the area of mobile carriers you had the sms
the 140 character 120 carat sms and it was great because everyone
agreed on the standard so as a consumer you got a lot of distributions and
interoperability but it was a very constrained format
and and when the industry wanted to add pictures to that format to do the mms
i looked it up and i think it took from the late 80s to early 2000s this is like
a 15 20 year product cycle to bring pictures
into that now once that entire value chain of
creation and consumption got wrapped in one software stack
within something like snapchat or whatsapp
like the first week they added disappearing messages like then two
weeks later they added stories like the pace of
innovation when you're on one software stack
and you can you can you can affect both creation and consumption
i think it's going to be rapid so with these streaming services we now for the
first time in history have enough i hope people on one of these
services actually whether it's spotify or amazon or apple or youtube
and hopefully enough creators that you can actually start working
with the format again and and that excites me
i think being able to change these constraints from 100 years
that could really that could really do something interesting i don't i really
hope it's not just going to be the iteration on on the same thing for the
next 10 to 20 years as well yeah changing the creation of music a
creation of audio creation of podcast is a really fascinating possibility i
myself don't understand what it is about podcasts that's so
intimate it just is i listen to a lot of podcasts i think
it touches on a human on a deep human need
for connection that people do feel like they're connected
to when they listen i don't understand what the psychology of that is
but in this world is becoming more and more disconnected
it feels like this is fulfilling a certain kind of need
and uh empowering the creator as opposed to just the listener
it's really interesting that's a this i'm really excited that you're working
on this yeah i think one of the things that is inspiring for our teams to work
on podcast is exactly that whether you think like i
like i probably do that it's something biological about
perceiving to be in the middle of the conversation that makes you listen in a
different way it doesn't really matter people seem to
perceive it differently and uh there was this narrative for a long
time that you know if you look at video everything kind of in the foreground it
got shorter and shorter and shorter because of financial pressures and
monetization and so forth and eventually at the end there's always
like 20 seconds clip people just screaming
something and and uh i'm really i feel really good
about the fact that you you could have
interpreted that as people have no attention span anymore
they don't want to listen to things they're not interested in deeper stories
like you know people are people are getting dumber but then podcast came
along and it's almost like no no the need still existed
once but maybe maybe it was the fact that you're not prepared to look at your
phone like this for two hours but if you can drive at the same time it
seems like people really want to dig deeper
and they want to hear like the more complicated version so to me that is
very inspiring that that podcast is actually long form it
gives me a lot of hope for for humanity that people seem really
interested in hearing deeper more complicated conversations
this is uh i don't understand it it's fascinating so the majority
for this podcast listen to the whole thing this whole conversation we've been
talking for an hour and 45 minutes and somebody will i mean
most people will be listening to these words i'm speaking right now you
wouldn't have thought that 10 years ago with where the world seemed
to go that's very positive i think that's really exciting and
empowering the creator in there is is really exciting
last question you also have a passion for just
mobile in general how do you see the smartphone world
this the digital space of uh of smartphones and just everything
that's on the move whether it's uh internet of things and
so on changing over the next 10 years and so
on i think that one way to think about it
is that computing might be moving out of these
multi-purpose devices the computer we had in the phone
into specific you know specific purpose devices and you know it will be ambient
that you know at least in my home you just
shout something at someone and there's always like one of these speakers close
enough and so you start behaving differently it's as
if you have the internet ambient ambiently around you and you can
ask it things
so i think computing will kind of get more integrated and we
won't necessarily think of it as as connected to a device in the same
thing in the same way that we do today i don't know the the path to that maybe
we used to have these desktop computers and then we partially replaced that
with the with the laptops and left you know we
had desktop at home and at work and then we got these phones and we started
leaving the the laptop at home for a while and maybe the
maybe for stretches of time you're going to start using the watch and you can
leave your your phone at home like for a run or
something and you know we're on this progressive path where
you i think what what is happening with the voice
is that you have an you have an interactive
interaction paradigm that doesn't require as large physical devices so i
definitely think there's a future where you can have your your airpods and and
your watch and you can do a lot of computing and
i i don't think it's going to be this binary thing i think it's going to be
like many of us still have a laptop we just use it less
and so you shift your your consumption over and
i don't know about ar glasses and so forth i'm excited about i spent a lot
of time in that area but i still think it's quite far away
ar vr all yes vr is is happening and working i think the the recent
oculus quest is quite impressive i think ar is further away at least that type of
ar i think but i do think
your phone or watch or glasses understanding where you are and maybe
what you're looking at and being able to give you audio cues about that or you
can say like what is this and it tells you what it is that i
think might happen you know you use your your watch or your glasses as a as a
mouse pointer on reality i think it might be a while before i
might be wrong i hope i'm wrong but i think it might be a while before we walk
§09Transcript · 1:44:12–1:46:59
around with these big like lab glasses that
project things i agree with you there's a it's actually really difficult when you
have to understand the physical world enough to
uh project onto it well i lied about the last question uh
because i just thought of audio and my favorite topic which is the
movie her do you think
whether it's part of spotify or not we'll have
i don't know if you've seen the movie her absolutely
and uh their audio is the primary form of interaction
and the connection with another entity that you can actually have a
relationship with actually fall in love with
based on voice alone audio alone do you how far do you think that's possible
first of all based on audio alone to fall in love with somebody
somebody or well yeah let's go with somebody just
have a relationship based on audio alone and second
question to that can we create an artificial intelligence system
that allows one to fall in love with it
and her him with you so this is my personal
personal answer uh speaking for me as a person
the answer is quite unequivocally yes on on both i think what we just said
about podcasts and the feeling of being in the middle of
a conversation if you could have an assistant where
and we just said that feels like a very personal setting so if you walk around
with these headphones and this thing you're speaking
with this thing all of the time that feels like it's in your brain i
think it's it's going to be much easier to fall in
love with than something that would be on your screen
i think that's entirely possible and then from the you can probably answer
this better than me but from the concept of if it's going to be
possible to build a machine that that can achieve
that i think whether you whether you think of it as a if you can
fake it the philosophical zombie that it assimilates it enough or it somehow
actually is i think there's it's only question if you if you ask
me about time i'd have a different answer but if you say i've given
some half infinite time absolutely i think it's just
atoms and arrangement of information well i personally think that love is a
lot simpler than people think so we started with true romance and
ended in love i don't see a better place to end beautiful
gustav thanks so much for talking today thank you so much it was a lot of fun
it was fun
you
you
you
you
you
you