Automatic music playlist generation via simulation-based reinforcement learning
research.atspotify.com
research.atspotify.com
I'm never going to be satisfied with Spotify's recommendations until it's easy for me to curate the input to the recommender.
As things stand, to prevent Spotify from giving me crap recommendations, I have to keep secrets from it! If I want to listen to Led Zeppelin once but don't want Spotify to think "oh I should recommend classic rock from now until the end of time", then I have to go to a different app (YouTube or whatever).
There's a "private session" feature in Spotify which allows me to prevent searches and listens from entering into history, but Spotify really really doesn't want me to use it — it's cumbersome to access and it gets turned off automatically after a few hours.
EDIT:
The satisfaction achieved by whole-account recommendations determined by algorithm described in this post is going to be crushed by some app which gives me a UI where I can say "give me recommendations based on these N arbitrary things". Not just "this one artist" or "everything in my history". Let me design my own recommended channel.
I’ve been longing for a “kids mode” for a long time.
I seem to remember that Spotify used to have a heavy focus on employee-operated playlists that curated new releases, though I didn't really take advantage of them at the time. The newer (cheaper) algorithm-driven ones don't really cut it.
Might also depend on who else is in the room or car. If I'm playing music for myself my selections will be different from what I'd play if my wife was listening, because our musical tastes are different. Or a group of friends, etc.
I've never found that recommendations or automatic playlists are very good, on any of the services, though my experience is mainly with YouTube music.
The outcomes of our user model are primarily consumption-focused and summarize the probability of user-item interactions. Specifically, CWM has been optimized for three targets: completion, skip and listening duration. The reward for the agent is computed as the sum of the probability of completion for each track in an episode.
…
All policies take as inputs features of the user who made the request and the pool-provided set of tracks with their associated features. The goal of each policy is to select and order a list of tracks from the pool that maximizes expected user satisfaction, which we measure by counting the number of tracks completed by the user.”
Although, I suppose this line of thinking makes sense for ad-supported streaming, which is likely used by the majority of users.
The most expensive way for you to listen to spotify is to skip the song every 31 seconds.
How would you measure user satisfaction?
I’m fairly open to music, I’ll likely listen to a song once if it’s on a playlist. But that doesn’t mean I liked the song, only that I gave it a shot. It certainly means I liked it more than the song I just skipped, sure, but it also doesn’t mean I necessarily want to hear it again.
So you don’t add them to your library?
I think the real point to be made here is that this is part of the inner workings of the system, that most users of the system are unaware of. Hell, this article and the ensuing discussions do not leave it completely clear how much of the system works. Like, I despise that when I create a new playlist with a name, it recommends a bunch of tracks based on the name of the playlist. Sometimes that'll be tracks with words in the name of the playlist in their name or some other odd metric, like it'll add songs from an artist that has a song that happens to be the name of the playlist. If you don't puzzle this out for yourself, you're possibly creating a very UN-optimized playlist for yourself.
The bigger issue I have is with “number of times played.” I listen to some comfort-food music over and over again, but the most rewarding music I listen to is challenging in some way and maybe isn’t something I listen to over and over.
Basically, Spotify’s metrics aren’t my metrics.
This is another place where Pandora really set themselves apart, the Music Genome Project. Any given track that went through curation has a (possibly very) large set of attributes assigned to it. This song you liked, it has a heavy bass-line, noticeable amount of shuffle, light drums, syncopated rhythms, etc. That's far better (to me) than "you might like other songs by this artist" or "other listeners of this artist also listen to", where the last one gets really sketchy when there's not a lot of listeners for the artist.
I'm also curious how it treats listening to a track on a Spotify station that is mixed, where they transition in to and out of the track late or early, so you won't hear the full track, does that still count?
Heck, sometimes I'll get almost to the end of the track and skip to the next just because the track has a long tail and I want to get to something that has more energy, not the dwindling remains of the rhythm or some soft piano fade out at the end of a 130 BPM track that had a lot of energy throughout most of it.
If they're going to make all of these sometimes seemingly arbitrary judgements of whether or not I like something based on these weird things like "there's a common English word in the playlist title that also shows up in all these song titles we're going to recommend to you", at least a list or chart of how it works somewhere would be nice, so I can make more effective use of the system.
I've felt this exact same sentiment and can't help but wonder if it's them being partially afraid of Apple Music in some way.
Since Apple Music launched with "add to library" as their shortcut action and "love" buried a bit.
A multi billion dollar company that fails so hard at this.
The best recommendations I got were from Pandora, unbelievable good and on point and that was 15 years ago. I can not use them anymore in Europe so I m stuck with Spotify.
Maybe this GPT4 based project will give me better recommendations.
If I like a track what exactly am I liking? The genre? The artist? Only specifically that song? Some maths that approximate a certain properties in the track? Some combination? How does liking song X influence Spotify's recommendation system on average? I have experienced the same as other posters WRT listening to a genre for a session only to have Spotify fixate on that genre until I've skewed the average in another direction. Small hack I've found if you make a playlist with a large and diverse sample size you can nudge their playlist enhancer to stay with the mood you are going for.
On granularity, I would want to be able to say:
Let x = track, genre, artist
I don't like X, never play it.
Exclude X just for this listening session.
Skew toward more X
I'd also imagine this would produce much richer data for future recommendation systems. If Spotify were to train on the data I've generated on their platform it would end introducing bias by the nature of me working around their systems and still not getting what I want, granted I may be an outlier.
Edit: Formatting & typo
0/10 indeed
Sure, you can claim that there's always the choice of not doing what the AI suggests, but that'll not apply to the kids growing up in a world where they're being breast-fed literally everything, including their playlist.
And their reason will be "because it's good", which is related to "because it's fun". When AI is perfectly pleasing, people will be fully enslaved by fun.
Fun already seems to be the driving factor for most people, so I'm not sure how to find anything positive in any of this.
Disclaimer: At the peak, I had around 300 gigs worth of language models stored on my machine. Now I'm down to around 120. :p
What are we doing here folks? Training models to match other models? Why? That’s good enough to get into KDD?
- the offline model data has a non-sequential world model
- the offline model is a great starting point but slow to react to realtime changes in user preferences
- the in-streaming of new data points makes the reward prediction inherently sequential
- hence you could use a cheap fast online model for fast reward prediction updates, that derives from a good starting point
The big question is: what do you do to adapt to just one new data point? You're trying to learn theta' := train(old data ++ new data) ≈ train(old data) ++ dTrain(new data) ≈ theta_offline ++ dTheta(new data)
If you change the offline model: well it's a big base model and there are always M users with N new data points coming in. Rerouting all training into the same common model theta, dynamically and live, in response to vast amounts of incoming data, while balancing batching-latency overheads or catastrophic forgetting or reversion to the mean preference, would be tricky.
If you change the online model: the delta per-user can be cheap to track individually with essentially increment-only/append-only per-user data structures. You can get away with approximating user_theta' ≈ offline_theta + delta(train(new data)) ≈ offline_theta + online_train(new_data). This can be easily parallelized and personalized and zero latency.
It's a good achievement.