Hacker News Reader

Comment by toxik | Hacker News Reader

Edit: this is a somewhat OT rant.

Netflix’s recommender system is hands down the worst I have ever seen. Every single thing I watch, it suggests The Queen’s Gambit and two other random Netflix productions.

Even if I watch the first of a trilogy (LotR, for example). How can they be so terrible at this?

The categories in the main browsing view are also hysterically arbitrary. It kind of looks like a topic model with back-constructed titles for the topics.

Finally, they replaced the ratings with “% matching”. I guess so they can recommend their subpar productions even if they get low ratings.

View on HN

>Finally, they replaced the ratings with “% matching”. I guess so they can recommend their subpar productions even if they get low ratings.

That's always what ratings were. People didn't understand that (as you can see), so they changed it to make it more transparent.

https://www.businessinsider.com/why-netflix-replaced-its-5-s...

>Netflix’s star ratings were personalized, and had been from the start. That means when you saw a movie on Netflix rated 4 stars, that didn’t mean the average of all ratings was 4 stars. Instead, it meant that Netflix thought you’d rate the movie 4 stars, based on your habits (and other people's ratings). But many people didn’t get that.

It's not just that (obviously, since It's not even possible for me to rate a movie more granular than thumbs up/down)

What happened is that the notion changed from "predict a scalar rating" to "predict a binary satisfaction."

As parent poster noted, the effect of this is to push "3 star" and "4 star" acceptable shows to the user, instead of "5 star" great (in the user's view) shows.

Also, in Netflix's defense, users are horribly inconsistent in their expressed ratings (they'll rate a movie based on their personal mood at the time, and they'll binge shows they claim aren't 5 stars while ignoring their 5 star movies)

This was a sad turning point for me. We used to have a single streaming platform with an awesome library, a granular review system, and user reviews. You could easily take a quick look to read other users thoughts on a film. Now I have to lookup metacritic / reviews myself, and the NF recommendation is based on whether I've said something is 'palatable enough to watch, isn't terrible, but I'd never watch it again' (thumbs up). I've taken to only thumbs upping stuff that I particularly like to see if that's any better. It all seems to be the same from an uninformed end user perspective.

I remember they stated the reviews would still be available in some form for export.

It's no longer the Netflix of old imo.

I've wondered if the lack of this stuff is due to business contracts or internal product goals. Not having e.g. IMDB makes sense since it is owned by a competitor (Amazon; whether that's a USA Trust thing, who knows)

This is exactly why they switched from 5 stars to up/down/blank.

Not just the recommender, but the design is horrible for me. I can't rest my mouse anywhere, it auto-starts something or enlarges something, it's all too twitchy. When you're watching an episode, there's no navigation link from the play screen to the main page of the series. As if they don't want us to navigate the site, instead be led on their happy path.

I think so. This way, you don't easily arrive to the conclusion the items I want to look for all don't exist here.

Good news is that you can disable autoplay of previews [0]. Bad news is that it takes weeks to propogate this setting change to all clients. I had to wait about two weeks for my apply tv client to stop auto previews. I suspect the queue service is powered by snail mail.

[0] https://help.netflix.com/en/node/2102

That must be because of an excessive amount of caching on the client-side

You would hope that logging out and back in would refresh these settings.

It does not.

I long ago ceased to believe that the Netflix recommender system serves any other purpose than to fulfill the company's internal obligations to push favored content, depending on what it cost. Sadly, the same is now true for Amazon Prime, which is an even hotter mess.

I turned on "Super Wings" for my kid to watch on Prime Video which at first glance seemed to be a fairly decent Paw Patrol knock off, but then as I listened to the episodes in the background, I realized that the entire show is basically an advertisement for Amazon Prime in disguise. Seriously look it up, the entire premise of the show is people ordering packages and the "Super Wings" delivering the packages to the consumer as quickly as possible... .

Seems like you got what you expected... Super Wings is the corporate side of the Government-Corporate complex.

That's exactly what's happening. Source: you can imagine.

Maybe it's different in other countries (I am in the UK), but I feel as if there isn't enough content for a recommender system to even be useful. I feel like after browsing through the catalogue a few times, I have a rough idea of most things I would ever possibly be interested in. There's just not that much there. Either that, or the recommender system is working too well and I never see anything beyond what Netflix wants me to.

I have to agree. As far as I recall the Netflix search engine has never found the exact title I've been searching for. Even the recommendations it then shows me are so far off piste that they're not really even close to what I'm looking for.

At least with Amazon Prime there's a high chance they'll at least find the title one searches for and if I am really motivated I can pay a few bucks to watch it.

Netflix just draws a blank.

That's not to say I haven't watched some entertaining things on Netflix, but they seem much better suited to TV series than movies, and I almost feel like I found decent things to watch inspite of their recommendation system, not because of it.

An ML tool from Netflix? It feels like the last thing I'm likely to use.

>Netflix’s recommender system is hands down the worst

Until you log into prime video.

Can’t manage to give me a “continue watching last thing button”. That’s literally the most likely thing I want to watch. Also routinely suggest starting with S02 even though I’ve not watched S01.

Never mind machine learning some common sense would be greatly appreciated

Netflix actually organized a major machine learning competition more than a decade ago[1] with thousands of the best researchers trying to beat some internal benchmark by a few percent, for $1M prize, I'm wondering where did all that "learning" go.

My guess is that whatever rankings they currently produce maximize some internal revenue target and that target' user base is not me or you.

[1] https://en.wikipedia.org/wiki/Netflix_Prize

Yup. It just ends up irritating users though if the suggestions break basic common sense.

If I watched S01E04 yesterday I want to watch S01E05 today. The interface should be suggesting that PLUS whatever else the ML comes up with in addition, not instead of.

Prime is full of minor irritations like that make me wonder whether Amazon engineers dogfood enough

Their recommender system likely has multiple inputs that are under specific constraints and weightings. How well their recommendation algorithms work will probably never be known by the general public as we are force-fed a steady diet of whitelisted staff picks and promotional items.

So what's the point of making tools like Metaflow? Justifying salaries for their engineers I suppose?

I see this complaint about poor recommendations very often.

But the recommendations seem to work perfectly for me - I wonder whether that is the case for the silent majority?

The match % is usually spot on for me, and I've never seen it recommend any titles I've given a thumbs-down to.

For many of the complainers, Netflix is showing them something they would actually enjoy. How to communicate that is challenging. I remember Pandora’s generated stations having a similar issue years ago, where it would catch similarities between on-trend bands and bands that had fallen out of favor. It was right, but it could sometimes be hard to submit to the insight of the algorithm. Spotify is one that I think does this well as they understand how not to give a recommendation that insults or offends, even if it’s technically an accurate response.

Do you have a counter example of a streaming service that does recommendations better?

I read this as a condemnation of recommender systems in general.

That or a ceding of recommendations to marketing. That feels too cynical, but more accurate.

Recommendations maybe not, but I love that in HBO I can sort by IMDB score to find great movies/series that I haven't watched yet. I just watched Broadwalk Empire as an example, and loved it.

Netflix doesn't show IMDB scores, so I have to check it always independently...it sucks.

Back when I used Netflix primarily for DVDs, the recommender system worked pretty well for me.

Much later, when they switched to simple thumbs up/down, the recommender system was entirely useless to me. (Not merely because of the dumbed-down rating system; the recommendations were genuinely bad.)

For the time in between, I'm not sure if the degradation was gradual, sporadic, or not degraded at all.

It's pretty clear that their recommendation system and broader UX is designed, at least in part, to obfuscate how much content they have and how good it is. Back when it was DVDs and they had basically everything it was more about finding the next best thing for you. Now it's finding the next best thing they have and, preferably, something they own the rights to.

If you add dvd subscriptions you can still get the original Netflix recommendations back, including sorting by top predicted rating (which to me is eerily amazing). I used to keep the dvd subscription mainly for the recommender, with the delivered blurays considered an extra bonus for very rare movies you couldn't stream even if you wanted to pay.

Not a streaming service, but a lot of movie and TV databases tend to have decent recommendations for movies and shows similar to the one you're viewing. Although in this case, one could argue that the processing is offloaded to users who provide the recommendations.

The MovieLens recommender system works pretty well.

It’s not a home run for me, but I think Spotify recommendations are quite good. They clearly use some form of content based recommendation (extracting features from the music itself) blended with other methods. It seems to make an honest attempt at serendipity (songs/artists you may like but would otherwise be unlikely to discover).

I still think recommender engines should always enable some form of user tuning. If it doesn’t, then the recommender is a tool for services to control your behavior rather than the other way around.

Unless I’m missing this functionality somewhere (entirely possible), this lack of user-tuning has ruined Spotify recs for me, since I listen to entirely different playlists when working or meditating. I don’t check out recommendations to get the latest binaural beats or nature sounds, you know?

Although even before I started listening to Spotify while working etc, it seemed to have run out of things to recommend. My weekly discover playlist would be half things I’d already liked. So...who knows. But I miss the discovery functionality quite a bit.

I personally have a similar problem with the Spotify generated playlists, but I have persnickety preferences in electronic music so it often whiffs. But on the spectrum of recommendation engines it is on the side of honest effort (whereas Netflix is not). And for most users and musical palates I think it works great. Just yesterday my mom complimented me on my Christmas music DJ skills, but it was just the generated continuation of her own playlist.

Nothing really beats the recommendations of a human curator with exquisite taste, and just listening to new music nonstop and plucking out the gems as you go.

I like Spotify's recommendations and have found lots of great artists from it. But now I feel I'm in a kind of Spotify-created rut of listening to static groups of artists. I have mostly stopped listening to my daily mixes because of this.

It's also easy to theorize about "big media" controlling what I listen to, so I still feel the need to do my own exploring, even when I'm getting recommended good fresh stuff.

Spotify. Discover Weekly is very good, particularly considering they have only one chance to recommend with 30-40 songs, and quite a large universe to match on.

Not a streaming service but Google's Discover news is also very good (probably the best recommendations I have come across).

The basic problem is understanding WHY I liked something. If I watch Tintin because it’s a cozy throwback to my childhood, that does not mean I would like every single Studio Ghibli movie in my recommendations.

Similarly, if I play Blacklist in the background as basically noise, I don’t want to see a bunch of related shows. I guess I could give it a thumbs down but I only do that for actually terrible movies.

Also Spotify and Apple music seem to have okay recommendations

Music is different from movies and television though. There are beats and rhythms that are easy to identify, lyrics easy to analyze and artists that are roughly categorized.

I feel like that’s not true, music is hard to analyze. Movies have synopses, and are generally categorized by their cast alone. I could figure out LotR belongs together from these datapoints, easily. Not to mention the fact that people watch trilogies in sequence like 99% of the time.

Spotify's Discover Weekly has a pipeline straight into my brain and not only what I like, but what I will like in the future.

Netflix's biggest problem is not the recommendation engine, but their lack of content. If they allowed you to filter out the junk you don't want to see and what you've already seen, there would be hardly anything left. They've been losing rights to stream left and right. That's why there's such a rush to produce their own content in other countries. Some of that is halfway decent, but there's a lot of formulaic, repetitive stuff.

And while saying that, I appreciate their high level technical staff. These decisions are made by bean counters.

> Every single thing I watch, it suggests The Queen’s Gambit and two other random Netflix productions.

I always assumed those three were human selections, not part of the recommendations.

Like the paid ads on top of your search results.

> recommender system is hands down the worst I have ever seen.

In my limited experience, they are all dismal and best ignored entirely.

When it was easier to see something plausibly like a real user review, that helped I guess.

I don't think it's the worst, it's just more that you've made the assumption that Netflix wants a pure set of recommendations. Marketing and business drivers will always trump algo results, so what you're seeing are mostly artificial boosts given to globally 'hot' properties.

Can we please take these off-topic rants somewhere else? Lately, it's hard to find any insightful comments in the disucssion section of engineering or technical articles. Because the whole conversation is derailed by something unrelated off-topic rants. There are a lot of social media, forums and even HN submissions where everyone can rant. Please keep the technical submissions clean.

Fully agree with this. Yuck, Netflix recos are ultra bad.

Netflix often promotes new content by overzealously recommending it, the constant suggestion you see for The Queen’s Gambit is probably an ad.

I really think that if they gave up all the complicated algorithms and went with a simple algorithm out of the 90s we'd be much happier with the recommendations.

Do you have such an algorithm?

Netflix doesn't make money by showing people stuff they don't like.

Not for a lack of trying though...

They actually might... think of it like gym memberships. Sure you need a hook, but as Mandalorian showed, that can be a single exclusive tv show that doesn’t require much broadband cost. Maybe not the best solution long term but since when has that been the shareholders goal?

Reply on news.ycombinator.com