Comment by jacobobryant | Hacker News Reader

Comment by jacobobryant | Hacker News Reader

Parent Full thread

(Reposting, again, with permission from dang)

Hi, OP here. I started working on recommender systems in 2016 during my undergrad, specifically doing music recommendation (I was dissatisfied with the quality of Pandora/Spotify recommendations). I spent about five months in 2019 trying to make a music startup based on that. However, during that time I realized that there would probably be more value in having a really good general-purpose/cross-domain recommender system. i.e. if you're looking for something specific, use Google, and if you're not looking for something specific, use Findka. That's the vision anyway.[1][2]

More specifically, the benefits I see from cross-domain recommendation are:

- More data per user => better recommendations.

- More potential users => (eventually) better recommendations. For example, to get users for a podcast recommender, you have to find people who like podcasts above a certain threshold. With Findka, anyone who's interested in getting recommendations for at least one content type is a potential user. (And even people who aren't above that threshold for podcasts might appreciate an occasional podcast recommendation).

- Lots of potential applications. I'm particularly interested with trying to use Findka for social networking (opt-in of course). Data from Findka could be useful for dating, job opportunities, forming online communities, etc. This is more long-term, but I also think Findka data could be useful for search.[3]

The algorithm currently is dead simple. Just collaborative filtering without explicitly taking into account content type. So it's naively cross-domain. Since the data set is still small, there's no need for matrix factorization. I recompute the whole matrix every hour and store it in memory. See [4] for the implementation (it only took 30 LOC). That's a little out-of-date but the general approach hasn't changed.

For the tech stack, I'm using a Clojure web framework + deployment solution that I made.[5] It's like a self-hosted version of Firebase (I'm running it on DigitalOcean).

[1] https://www.ben-evans.com/benedictevans/2015/6/24/search-dis....

[2] https://www.aaai.org/ojs/index.php/aimagazine/article/view/2....

[3] https://news.ycombinator.com/item?id=23449754

[4] https://findka.com/blog/rec-sys-in-30-lines/

[5] https://findka.com/biff/

> I started working on recommender systems in 2016 during my undergrad, specifically doing music recommendation (I was dissatisfied with the quality of Pandora/Spotify recommendations). I spent about five months in 2019 trying to make a music startup based on that.

Hi! I'm very interested in this domain, I have long been dissatisfied with the results for music recommendation across almost every platform (and the respective apps, but I digress). So frequently it seems my own personal preferences are washed away in the breadth of ML and algorithmic recommendation systems, that no matter what I begin listening on, I will invariably find myself within the mainstream for that genre/artist. It also seems to me that many algorithms can't properly deduce that I would want songs centered around a specific year, and instead seem to draw from what's "popular" in the genre regardless of the time period the song is starting from.

As an example, I can begin a GPM station based on I Gotta Feeling, a song from 2009. When I start a playlist like that, I nearly never am rewarded with a song from that time period. I tried it just now, it immediately jumped to Roar by Katy Perry, a song from 2014 with an entirely different vibe.

I supposed my question is, how meaningful are the music recommendations Findka produces?

I would love to talk more about the music startup you were pursuing, and what you accomplished there, as well as any roadblocks you may have faced. I have been whittling down an idea for a music startup myself and can't help but wonder where that road led for you.

Also your site link is broken/not resolving.

I'd love to chat more about music etc. You can shoot me an email at hn at jacobobryant.com. Definitely a pain point I feel as well, though another reason I switched to Findka is because I figured it would be a lot easier to bootstrap it. The music industry is pretty tough to get started in.

As far as the current music recommendations; it's nothing intelligent. Just pure collaborative filtering, i.e. it has no understanding of "this is a rock song" or "this song is from 2005" or even "this is a song". Right now, the algorithm just sees a bunch of URLs and their rating data. I am interested in making the algorithm more intelligent over time though.

Could the site not resolving be an issue with your network? It Works For Me, and I'm getting plenty of traffic right now.

Pretty surprised that the music recommendation results you've been experiencing have been quite poor. Your feedback would have been invaluable in retraining the algorithm to make recommendations that suit your preference. Feels like a lost opportunity. Moreover, I don't know if much thought around self-discoverability/popularity has been factored into the design of the recommendation system - ideally, the recommendation system should surface interesting things that I like that I wouldn't necessarily discover on my own.

I've wondered about this a lot. It seems to be hit or miss--some people love e.g. Spotify's discover weekly/daily mix playlists, and some people hate them. I suspect that some users are outliers and it's hard to give them recommendations, and maybe tuning the system for them decreases the performance for other users. Also there may be limited business value in optimizing for those users anyway vs. pushing popular music.

I suspect you’re right about trading off between pushing popular music vs tuning for outlier users. Feels like there are too many smart engineers to miss it. Plus you’re building a personalisation system which should be personalising things for individual users by default.

I do think the idea has a lot of merit. I've seen collaborative filtering work very well in movie recommendations, and I've observed in real life how often social groups overlap with multiple similar interests.

I think that scaling will be a challenge, both computationally and gathering enough data to be meaningful. The biggest wins, I suspect, are when you can infer people's preferences incidentally (say, from watching what podcasts they actually talk about on social media) rather than from their self-reported preferences, where conscious intentions often override true emotions.

But I'd love to see this work. I know there are things out there that I'd enjoy but don't manage to connect with.

I've thought a fair amount about inferring preferences (i.e. explicit vs. implicit feedback). Particularly for music, tracking skip data and feeding recommendations to the music player would make it a lot more convenient. My plan is to integrate more deeply with specific content types as time goes on/after the business is working.

This sounds a lot like Google, which is in a sense a giant cross-domain recommender system.

Yeah. I think of recommender systems as one level of abstraction/automation above search engines: the only difference is that with recommender systems, you don't have to type in a search query.

Very cool and nice vision statement!

Quick heads up, the book links aren't working for me.

Thanks for the heads up. Turns out the default url that google books returns often gives a 404. I fixed that last night but haven't updated all the previously imported books yet.

Reply on news.ycombinator.com