Hacker News Reader

Comment by tempestn | Hacker News Reader

I'm apparently in the minority, but I long for a return of the 5-star ranking. I really miss that about itunes, being able to rank each song, and set up smart playlists based on genre and how much variety (how deep in the star rankings) I wanted.

I'd have to think an intelligent recommendation engine could do a ton more with star rankings than a simple up/down too. 5=love, 4=like, 3=ok, 2=dislike, 1=hate. It could even be optional; they could just treat like and dislike as 4 and 2 or something, and let people who want more precision use them.

View on HN

It needs more than that. I often hesitate to rank because my preference often varies by time and mood and setting. I might not dislike a song, but don't want it played when I want something mellow to fall asleep to, or when I'm happy or when I'm sad. Having a given song show up in one context might irritate me because it breaks the mood while I might love it another time.

A good recommendation system for music needs to take into account proximity in play order to other songs you've ranked when you rank it, and their tempo and genre, time of day, etc.

The more information the better, thing is platforms want to sell their content first and foremost. This removal of user input was pushed by the business so they could push content regardless of personal interest.

Ever wonder why Netflix keeps on pushing bad content ?

One reason to keep on pushing bad content despite knowing the user will not watch them is to hide the fact how small their catalogue is and shift the blame onto user's particular taste.

Do you have any evidence for that claim, or is that pure speculation on your part?

I don't have any written or direct evidence, however I worked for a public service broadcaster, who regularly dealt with Apple and Netflix, and the feedback from both was that star-rankings were driving a lot of royalty payments and user choice (people would ignore poorly ranked content). The impact of this was that low performing content providers made less money. By removing the collective scoring, the company would implement their own recommendations algorithms, which gave them the ability to both deliver 'better' recommendations based on what the user consumed and spread the royalty payments more broadly. If what you wanted was to see what everyone else was watching/listening to though, that's not in the interest of their business. Anecdotal and third hand but I believe the people that relayed this to me.

That makes sense, thanks for elaborating! I could imagine a different solution where instead you just don't report the overall star-average, to make it a harder metric to game, but still get to have a stronger feedback signal for your recommendations.

> "Netflix Ditches 5 Star Rating System, Is Amy Schumer to Blame?"

> "This past week, Netflix officially debuted their new Amy Schumer stand-up comedy hour The Leather Special. And it was instantaneously met with negative reviews. Some claim that Schumer's biggest critics got on Netflix and purposely drove down the rating of the special, some without even watching it. Schumer herself blames the 'Alt-Right' for sabotaging her latest effort. Now, it is being announced that Netflix is ditching its five-star rating system for something much more streamlined, a rating system that owes itself to the legacy of the late Gene Siskel and Roger Ebert. Did Amy Schumer have anything to do with this new rating system? Probably not. But once in place, it will certainly help her Leather Special find a more appropriate audience."

https://movieweb.com/netflix-cancels-5-star-rating-system/

'Probably not', but it sure seemed like coincidental timing to me.

I can’t even imagine the senior product and data people I’ve listened to deciding to change this based on one artists experience.

I'm pretty sure stars were discontinued because the best proxy for "what you will spend time watching, is what you spent time watching" and not your curated ratings. As in: the datapoint they funneled into their models was "time spent on video X" and not, "rating on video X".

Business of course wants to keep churn low, and they think time spent on the site is the best way to do that.

That (and "people mostly don't use ratings beside 1 and 5") is at least the reasoning I've read every time I've seen this topic mentioned in blogs/talks.

I do also feel Netflix is pushing it's in-house content way more these last few years, to the detriment of their recommendations though.

What is Apple selling to have so little settings in MacOS? ;)

There are two issues with 5 star ranking. One the more thought required to give feedback the less likely users give feedback. More interactions is very helpful for ml so minimizing the decisions needed by a user helps a lot. Great ui/ux for engagement aims for signals to require as little action from the user as possible. As a side effect short content is very nice as finish/playtime become strong signals that require a user to do almost nothing.

The second issue is a lot of people use it like 1/5 system. While yes some users use more of the intermediates and you can adjust on per user basis for that feature it’s just not as useful as you’d expect.

Of course any decision like this I recommend a company ab test and I’m just summarizing past experience in this area.

I had a similar discussion at a prior job on having creators add content tags to there submission. The main avoidance was while good tags would be nice for recommendation, if tags are required less content would be submitted and that hurts the platform.

> One the more thought required to give feedback the less likely users give feedback.

That is great! This means the people giving feedback would tend to be power users that are able to giver higher quality feedback.

> As a side effect short content is very nice as finish/playtime become strong signals that require a user to do almost nothing.

Perfect! It is clear by now that users vastly prefer short content that gets to the point quickly.

So basically 5 stars ranking would vastly increase the quality of the content but that is bad, how?

The only issue I see is (political) vote brigading. Maybe set additional hurdles like minimum account age and the like for voting. Strong content moderation could help. I guess there is no silver bullet for that issue.

Your first point isn’t true because most of the 5 star usage boils down to 1/5 stars and not 2-4. There are some users that use whole range and the info does have value. The main problem though is while that system may be better for certain users, it will be worse for the entire user base as you want a system that works well for both casual and power users. Really you want a system that works better for casual users as most users on typical system like YouTube/spotify/tiktok/etc would be casual users. And while having model distinguish between user types is possible, ui wise you aren’t going to want to have both thumps up/down as an option and 5 stars. A toggle is possible but even for users that would use full ratings, a toggle is fairly unlikely to be used.

Even for power users it may not be net benefit for them. Recommendations are not based solely on that user’s ratings but all ratings. If power users give better signal but total quantity of signals decreases then it’s likely that recommendation quality drops for both casual/power users. Making up guess numbers a 5 star system is likely to get you 3x less data points and many of those data points are of similar quality anyway (users that only rate 1/5 stars). Worse signal that has higher volume often leads to better model.

My argument was based on the premise that votes from power users are more valuable as they better at judging the quality of the content. While the quality is partly subjective, it is not entirely so. Meaning power users, having more experience, know better what casual users should watch and would like then the casual users themselves. It is in the interest of the casual users for them to not be able to vote/their vote to count less.

I later suggested ways to make the hurdles for voting higher, which would fix the problems you mentioned. Required account age, maybe even how much content you need to have watched. Honestly, we could just filter out the people that only do 1/5 star votes. (With niche content of course there would be the issue of having no/just a few votes but voting would not be the only metric anyway.

I am very heretical towards common held UX believes and think trying to design for the lowest common denominator just results in all around awful experiences for everyone. We should rather strive to empower users.

> My argument was based on the premise that votes from power users are more valuable as they better at judging the quality of the content.

That is quite unlikely. What you call power user is someone distinguished by willingness to rank. It does not make that person better representation of all users or be more knowledgeable off quality or more experienced. Just more judgy.

It means you will be recommended based on tastes of special minority and likely alienate the rest of users in the process.

> My argument was based on the premise that votes from power users are more valuable as they better at judging the quality of the content.

As someone responsible for designing and incorporating end-user activity/feedback into ML, I have to tell you this is exceedingly unlikely.

The number of power users, especially on a large consumer app like Spotify, is dwarfed by the broad userbase (I estimate ~1% of Spotify users at most qualify as "power users").

Would I rather have power user-only feedback, or 100x the feedback? Definitely definitely 100x the feedback from all users.

I don’t think counting there vote less is useful. Low quality signal that is plentiful is very often better than sparse higher quality signal. My past experience is working on recommendation systems at tiktok/snap. Having a lot of weak signal from casual users is extremely helpful for power users. Recommendations are often based on large amount of basic engagement data. I think a large aspect to TikTok’s system being considered strong at recommendation is because Ux is designed for very frequent simple signals.

I can also say working at companies like that generally any system changes require an ab test and long running back test. I’ve generally seen companies move from 5 star to simpler like/dislike system and those changes would have been ab tested and found that user experience is overall better. I don’t know for specific systems whether overall better was across all users or some users.

I think these issues are more significant for public rankings. When I'm ranking songs, it's to curate my own listening. So if I only want to use 1 and 5 stars, fine—no worse off than a thumbs up/down system. But for those of us who do want to be more precise, it just gives more flexibility.

The systems I worked on were mainly for user curated recommendations and not public rankings. Your rating is only small part of your recommendation. A large amount of your recommendation is finding other users with similar ratings on similar content to you. If 1/5 star system leads to less ratings from other users, then system's ability to find similar users/similar ratings become worse and can negatively impact your recommendations.

I can't rate using 5 stars, it's either like, dislike or don't care, so 1 star 5 stars or no rating. With Tidal that's either listen to <30 seconds, skip or maybe listen through the full song, maybe fave or go back and replay. So they don't even need a dislike button. They just need to look at user actions if the algorithms are really optimized for tailoring content to the user's preferences, rather than pushing sponsored content in order to make money really fast. I don't even look at their recomendations (they're BS), but I do play artist radio if I happen to like something. That's how I found about Toshiko Akiyoshi a few days ago while listening to some artist's or track's radio.

I've thought about this in the past and figured something geometric could better capture engagement

-1 -> dislike, +1 -> like, +2 -> favourite, +4 -> heart

Labels are arbitrary. How might this weighting affect learning?

There is something I've never seen in a voting system before, but I'd love for it to take off.

Slice open an orange along its equator. Looking at this slice you have a shape kind of like an *

Each of these wedges can have a different tag. Those wedges facing up would have a positive value, and down a negative.

The site the orange slice voting system is used on can easily decide the number of wedges that make sense and the tags associated.

Instead we get binary or unitary choices that don't really mean much.

Edit: Image for example

https://i.imgur.com/0uVQl3s.jpg

Itunes (well I guess it’s Apple Music now) does still support 5-star rankings, it’s just hidden behind a context menu. I still have a Smart Playlist (can you even make those anymore?) of my 4-star and 5-star songs that serves as my daily shuffle rotation.

Yeah, I still use it for listening to my own music, but I'd love to have the same thing for streaming. (Maybe Apple Music supports that?)

I rate songs from Apple Music (that I’ve never downloaded to my library) and they do appear in my rankings-based smart playlist.

Apple Music still has star ranking, although it’s a bit buried away.

It’d be better as:

+2 love

+1 like

0 don’t mind it

-1 dislike

-2 never again

It's called a Likert scale: https://en.wikipedia.org/wiki/Likert_scale

thanks!

Reply on news.ycombinator.com