Transformers in music recommendation

The reason this simple mechanism works so well is that it gets rid of personal biases and instead taps into a community of listeners listening to the same stuff. Confirmation bias is the core issue here. I don't want confirmation bias. I want my biases challenged with new things. Not randomly new but based on what others are listening to that listen to similar things. And not just randomly based on everything I listen to but on specific things that I'm playing.

Vector similarity of artists could be an interesting angle. But it would probably risk pulling out a lot of cover bands and imitators. You want stuff that is close but not too close.

It requires the music to be already present, however, so not ideal for finding new music.

Considering the pool of music on the radio when I was a kid, that's a reasonable hit rate in my mind. I'm not sure I would cope with an influx of music larger than that.

Genre, tempo, key, vocalist sound, instruments, and so on. These might all be relevant in different recommendations, at different times, in some particular order depending on the user. The music-content in effect only serves to align tracks along lines in the embedding space.

Algorithmic playlists I've not found useful. The Apple Music "create a station from this song" feature is more or less broken imo -- i get so much of the same same same stuff

Apple Music is notably better – and has the benefit of not funneling your money to the likes of Rogan – and the recommendations will be fairly good within a genre but it does overweight your library a bit (I wish it had a “I’m looking for something new” / “familiar” toggle).

I am curious what Rdio did differently as I had a very good success rate with their suggestions and it seems unlikely that there was some secret sauce nobody else has been able to figure out.

Related question: I wonder if identical twins are good at recommending each other music

That's the thing with transformers, right? It doesn't actually "know" anything about its inputs.

The embeddings are learned (initialized to random).

They took all user generated playlists and projected the songs into vectors where songs that appear together on playlists are closer and songs that appear less often are farther.

It’s likely changed a lot since then, but it seemed like a pretty straightforward clustering system at the time.

It uses data about overlap in listenership between different artists to determine which artists are related to which others and how. The artists serve the same role as words in sentences.

My experience is that the best music is found randomly. I like so much, I don't even know what I really like. Even what I like is always changing. I need to listen to ton of random things I don't like and I will find a small amount of gems. The absolute gold though is finding songs I didn't even know I would like.

The algorithmic version of sifting through records at a record store for a music lover is random. Random with an easy way to play the next song.

All these recommendation systems are just Satie's musique d'ameublement generators for non-music lovers. Furniture music generators, music to play during a dinner to create a background atmosphere for that activity.

Most of the reasons people like music, or fictional movies and books, is personal, emotional, subjective, and difficult to articulate. You wouldn't know what data to collect. You're better off just asking them to rate song, movies, or novels out of ten. You can then compare their ratings with other people's, and what you'll find is there are clusters of people who rate things similarly (and others who rate things differently), and that the ratings they give overall somehow capture their feelings about whatever they listened to, watched, or read. (Source: I developed a movie recommendation system which predicted ratings reasonably accurately.)

Of course, if you just have sequences of user actions, like in the article, your recommendations won't be anywhere near as accurate.

Years of experience have proven that you can get quite far with pure collaborative filtering—no user features, no content features. It's a very hard baseline to beat. A similar principle applies to language modeling: from word2vec to transformers, language models never rely on any additional information about what a token "means," only how the tokens relate to each other.

If I try to play any music from a historical genre, it's only about 3 or 4 autoplays before it's queued exclusively contemporary artists, usually performing a cheap pastiche of the original style. It's honestly made the algorithm unusable, to the point that I built a CLI tool that lets me get recommendations from Claude conversationally, and adds them to my queue via api. It's limited by Claude's relatively shallow ability to retrieve from the vast library on these streaming services, but it's still better than the alternative.

Hoping someone makes a model specifically for conversational music DJing, it's really pretty magical when it's working well.

> Plex Media Server uses a sophisticated neural network to analyze each track in the music library, cataloging a wide variety of characteristics of the track. Think of it as things like female vs male, vocals vs not, sad, happy, rock, rap, etc. All these various characteristic constitute a “Musical Universe” and the server is determining where that particular track exists within it.

> For the math-savvy, the Musical Universe consists of points in N-dimensional space. But what’s important is that this allows us to see how “close” anything in your library is from anything else, where distance is based on a large number of sonic elements in the audio.

I haven't tried it so can't speak to its effectiveness.

The problem with classification is that what makes a genre is not uniform. Some genres are defined by the way people sing, other genres are defined by the singers language, other are purely about the instrumentation or rhythms used, yet others mostly about the sounds and notes used etc.

But there are things like tempograms, tonnetz (tonal centroid features), chromagrams, spectral flatness/contrast/roloff, laplacian segmentation etc. And I guess feeding these into some neural net might give you interesting results.

Pandora and Rdio and others solved the exploit problems years and years ago.

Doing music similarity with Echo Nest was great back when it was public. I did a project in grad school with it.

it's interesting how this continues a trend in across music/audio tech, such as hipsters insisting "i liked the earlier work better" or audiophiles obsessing over amps from the 1970s.

Sometimes I'll bounce off of a given artist several times, over years or decades, until the right track catches me when I'm "ready" for it, and then I'll enjoy discovering their whole catalog. At that point, the affinity is durable. I would love to find whoever is my next (e.g.) Steely Dan.

Then don't play music. You are asking for something that only works in a dystopia.

Either the machine tries to understand "what I need now" from partial information and will not play me music when I want to listen because it thinks it "knows what I need".

Or the machine actually is hooked up to my real time health data and possibly brain activity to actually know what I need.

I definitely don't want to have a personal computing machine thinking it knows what I want and deciding for me in such a way, or to have such access to my internal state.

https://hardcover.app/askjules