I don't think Spotify has any kind of special algorithmic sauce, they just make it easy for people create and share playlists.
It amazes me how most of the other streaming services try to turn everything into an algorithm, via "special radio" stations, etc. And yet, the best way to provide a long tail for music still seems to be basic web 2.0 - let the people make the content for you.
For a solid year, any time I played a Daily Mix or let one of my personal playlists runout and it reverted to "recommended music", Neko Case was always in the first top 3 songs. Before that it was Iron and Wine. I don't mind either artist, but I have never chosen to play either one myself.
I do like Spotify's curated playlists though. They are always a better source of new music than the automatic playlists.
The main benefit of Spotify is that I can just bypass the algorithms, and go straight to people. And that's really where the "long tail" effect seems to thrive.
I had been switching between Apple Music, Spotify and Google Music in different countries. Currently I have settled on Google Music simply because Google practically abandoned Google Music so they are not so in my face with curated playlists and stations.
On the other hand I think that there is opportunity for good (like in really working) AI engine for music recommendations.
If the algo scripted a TV sitcom, it would be 5-6 mostly white people who occasionally hook up, break up, get married, struggle to conceive, and then everything eventually works out.
Same goes for music. People have a habit of liking what's in front of them. Few are going to want to make the effort to find something that's more than "good enough".
I had been clicking 'I don't like it' buttons hoping the AI will get it but no luck. So after 30 days I had enough and moved on back to Spotify.
So much for human feedback.
(Just kidding, but this kind of thing will be more common in the future, especially as we learn more about how little the brain knows about its own desires and motivations)
My sense is that is probably as good as it tends to get, unless you want to get incredibly creepy and start bringing outside personal information, e.g., associating that the 40-something living in Chicago may have an affinity towards Wilco. Personally, I'd rather deal with "meh" suggestions then have my music service track detailed demographic models of me.
I think they do that because it's the only way to keep people from switching to rival platforms. Spotify's selection isn't that different from Google Play Music/Apple Music/Amazon Prime Music etc. There's also little in the way of community interaction on those platforms beyond "sharing playlists".
SoundCloud at least has a system where you can comment on specific parts of a song. It creates a really interesting visualization of which parts of a single song people enjoyed the most.