Ask HN: Why do recommendation algorithms suck?

It's difficult to talk about a good or better algorithm without quantifying what that means. Youtube aren't trying to build a good algorithm. They're optimizing for a metric like time spent on the website. If your algorithm could beat their current solution in an A/B test, I'm sure that they would be up for it.

I’ve given up on content-based recommendation. The issue is that people’s interests in their topics and subtopics is primarily dictated by their individual circumstance and environmental real-world context. That is, if you are able to perfectly know the interests of an individual today, perfectly label the topic of all content, you’ll still give them incorrect content after a few days—-and with news article recommendations, by the next session of platform engagement (since they’ve sufficiently extracted the maximum value from the present topics by the nature of the session).

My idea allows for change of categories pretty much anytime. This user control maybe be able to circumvent the problem you're describing.

They’ll still have to sift through tons of no longer context-relevant content. You’re also expecting the user to rationally know his context in terms of your chosen categories, and worse, to put in the work to formalize them for your engine at every session.

There is no information in any content or a previous interest matrix which guarantees any real-world relevance in the next session. Instead, it’s likely the user exhausted the utility of their previous topic matrix. The effect is that the moment you determine content relevant to their context, they’ve already been sufficiently informed. I’ve tried algorithms far more advanced with ML/Lebesgue measure theory, which were technically optimal, but with practically laughable results.

I'd be interested to hear your thoughts on this: https://findka.com/blog/essays/ (a project idea I'm thinking about doing). I wonder if preference for essays/evergreen content (as opposed to news articles) would be less affected by day-to-day context.

I remember when the Netflix prize was underway, there were certain movies—Napoleon Dynamite was one of them specifically—where it was incredibly difficult to determine if someone would enjoy it based on their other ratings. And yet it was also enormously popular, so lots of people ended up reviewing it.

Honestly, read up on the articles discussing the Netflix prize, they tended to mention a lot of unexpected gotchas. For instance, it turns out, ratings aren’t really independent. If you rate something good, it will affect how you rate the next thing. Stuff like that.

Definitely not an ML person perspective:

One type of recommendation system is user-user recommendation, so it is taking stuff you've (A) looked at, comparing it to what other's have seen (B), then trying to find what from A that B might like from weighting features of each possible item to find the item that person might like most (and therefore, do the action you want them to: view, buy, etc)

So if you view Video 1 (action), Video 3 (action) and Video 4 (comedy).

And someone else views Video 4 (comedy), Video 2 (comedy) and Video 3 (action).

If we are only using genre as a feature, the system would want to recommend you Video 2 (a comedy you haven't seen) and it would recommend the other user Video 1 (an action video they haven't seen).

Your system is introducing new features that could be useful.

In my opinion, the reason why recommendation systems suck is either people learning how to game it and therefore have their stuff recommended when their item shouldn't be, the system not having enough data or the system just not being great at figuring out what you like due to bad weights.

EDIT: Fully re-reading this post, it seems my post is a bit off-topic.

With that said, ML is not a dirty word or has to be some magical thing. It is applying stats to a large dataset. It's not anything fancy nor does it have to use the new shiny thing.

ML can be as simple as: go through this list of users and videos, given these weights, what videos would each user likely like most?

I wonder why big platforms don't give more control over their recommendations compared to what I propose. Maybe it's stakeholders or they want to promote certain things regardless of user liking.

Also thank you for making tube and reddit.qnzl.co. There is a whole community at reddit.com/r/nosurf who really need those.

TikTok as an interesting take on recommendations compared to other platforms.

Instead of presenting a list of recommendations (like YouTube, Netflix or Instagram does), TikTok infers what it thinks you will like and directly shows it to you. Then based on how you react: if you like the video, follow the creator, watch it until the end, or swipe away, they know with much more accuracy the type of content you enjoy. This allows for more testing on their part too.

On the other hand, on platforms like YouTube, there are a lot of factors beyond the video itself that influences if a person even clicks on it (like the thumbnail, the title, the number of views, who the creator is), so if someone doesn't click on a video, that doesn't tell you if they would have liked it or not. Because of that their data to make recommendations isn't as accurate.

YouTube recommendations could be made enormously more useful with two trivial (really!) fixes:

SELECT * FROM videos WHERE NOT EXISTS (SELECT 1 FROM votes WHERE video_id = videos.id AND user_id = $user_id) AND NOT EXISTS (SELECT 1 FROM not_interested WHERE video_id = videos.id AND user_id = $user_id);

Aka don't recommend anything I've already voted on or already explicitly told you I'm not interested in. That would remove at least 98% of my recommendations.

Even without that improvement, though, YouTube is an amazing service, and the reliability is first class. I can't remember the last time it was actually down. Personally I suspect the YouTube team is just tasked with keeping things ticking over, and that they're too busy putting out fires and coping with tech debt to actually develop useful features.

Yeah, they probably just plain economize on compute there.

Considering the computation necessary to downscale every video I'd suggest this join would be a small drop in a big ocean.

Maybe there are some differences between offline and online compute like page response times, cluster specifics, database latency etc. There are no hard timing or effectiveness requirements for processing videos after all.

The recommender is not there to be useful to you. If it is, that is just a side effect. The recommender is there to make you watch more YouTube. Even if you've explicitly said you're not interested in something, that doesn't necessarily mean you won't watch it.

Believe me, the YouTube recommender has plenty of resources allocated to it, because it's what makes YouTube such a huge business.

If YouTube cared about being useful, they wouldn't constantly reenable autoplay after you turn it off. Autoplay turns back on because it makes you watch more YouTube.

That doesn't really explain why they re-recommend previously acted-on videos though. You won't watch them anyway so they basically eat screenspace that can be used for some novel videos with ads that you will watch with a higher probability, there is no real lack of those now.

People re-watch videos. If it's showing up in your recommendations, the recommender thinks you may act on it.

And youtube knows exactly what videos they re-watch often enough so it can clearly mind this stuff, plus they will probably use the search if they want to re-watch something specific anyway. It is no reason to re-recommend every one of those including explicitly uninteresting.

Or am I missing some research that shown novelty less effective at driving the clicks here?

I don't have research to cite about specifically whether novel recommendations get clicked more often. However, YouTube makes $15 billion in revenue. The recommender is responsible for an enormous fraction of that revenue. Google and YouTube have an enormous group of talented ML engineers. I am certain that they have tested changing the proportion of novel recommendations, and whatever proportion they use is the proportion that maximizes engagement.

Personally, I hate the YouTube recommender as well. It's particularly aggravating that I'm stuck with it and its lack of configurability even though I pay YouTube to make the ads go away, so they get no additional revenue when I watch more.

However, Netflix does many of the same things as YouTube, including autoplay, because they have found that autoplay makes you watch more, and if you watch more, you're more likely to renew your subscription.

Making big revenue and having best teams doesn't mean they do everything most efficiently and don't ever go the way of the lowest friction. I am really curious whether someone really tested it and found it unfeasible from a technical or psychological standpoint. The autoplay stuff I clearly understand but the recommendations' - don't.

I may have rewatched maybe 1% of the videos I've seen, so either I'm way outside their target group, the recommender isn't adapting to my behaviour, or 1% is the best it can do.

Also, based on my experience with GDrive support I really don't believe that Google has excellent engineers all over the place. Like virtually all big companies before them, once they get big enough the overhead of keeping the services from regressing takes up all the available energy.

It's more complicated to the point that Netflix offered a $1,000,000 prize for someone who could build a better one.

https://en.wikipedia.org/wiki/Netflix_Prize

Looking at the link. It seems that it was beaten and someone received the money. Now, this is my reason for posting my idea here : it's to get feedback on it. It seems to be rather unlikely that I would come up with something better than teams who are payed to come up with those recommendation algorithms.

However, what do you think of the proposed idea. Maybe in theory it looks better but until I A/B test it I might never know if it's really better than what's currently out there.

I believe that the Netflix Prize was fundamentally flawed. They have access to much better information than user rankings, especially for the online service, things like "started but not finished" or "watched all of it the day it came out" and "watched more than once".

That's much more honest feedback than user's star rankings, which can be skewed by a lot of factors: users failing to rate things, rating based on what they feel they should like rather than what they actually enjoy, the psychological anchor of the suggested ranking. It's not surprising that their existing work got most of the value out of that data.

They have much better data now, and I'm sure they're not letting anybody get so much as a glimpse.

I think there can be some additions:

1) for user to be able to manage those percentages directly or using various pre-made profiles and maybe allow some randomness mixed in;

and

2) "exploraton mode" where topics with content that is "disliked but not conflicting with the core values", "disliked but they may try to convince me here" and "initially neutral, unknown or not entirely irrelevant to them" are suggested.

I also think that attempting to extract the core values as hidden variables and using them to predict, or guess from oher users and external news, the future direction of their evolution can also be useful.

On a surface level I would like this better. It's not a black box, and seems like a decent spread.

My question would be how do you get all of your content classified in a reasonable way?

If I look at websites like Quora or Medium they usually ask you to select broad categories. I haven't thought deeply about it but tags that are already used on YouTube and Medium seems to be the most obvious to classify the content.

Do you have an idea in mind?

I don't have an idea other than tags, I was curious if you did.

Tags seem like a good idea, the only thing that worries me is that they would be easy to abuse. That being said, I'd be willing to bet it would come out better than what we currently have.

If you look at YouTube there are tags but also the category that you can select when you upload a video (Lifestyle/How-to and so on) so maybe we could simply use that along with tags to get accurate classification.

However, if I made a new video platform (Daunting task) I would penalize content creator who purposefully mis-categories their content.

Seems like a decent proposition. If you wanted to hack around with the idea invidious might be a good place to start.

That should cover your frontend, so you could focus on the recommendation scheme.

Edit: If you do decide to play around with it and decide to open source it, could you send me a link to your repo? I'd be interested to watch, and possibly contribute here and there. My email is in my bio.

Another reason you didn't list is that it is surprisingly difficult to generate recommendation algorithms that don't empower hate speech. As an example we had to update the filters downstream of the main recommendation model dozens of times over the course of a week to stop Epstein and Pizzagate "researchers" from connecting with each other during the last election